关注 spark技术分享,
撸spark源码 玩spark最佳实践

AggregationIterator — Generic Iterator of UnsafeRows for Aggregate Physical Operators

AggregationIterator — Generic Iterator of UnsafeRows for Aggregate Physical Operators

AggregationIterator is the base for iterators of UnsafeRows that…​FIXME

Iterators are data structures that allow to iterate over a sequence of elements. They have a hasNext method for checking if there is a next element available, and a next method which returns the next element and discards it from the iterator.

Table 1. AggregationIterator’s Implementations
Name Description

ObjectAggregationIterator

Used exclusively when ObjectHashAggregateExec physical operator is executed.

SortBasedAggregationIterator

Used exclusively when SortAggregateExec physical operator is executed.

TungstenAggregationIterator

Used exclusively when HashAggregateExec physical operator is executed.

Note
HashAggregateExec operator is the preferred aggregate physical operator for Aggregation execution planning strategy (over ObjectHashAggregateExec and SortAggregateExec).
Table 2. AggregationIterator’s Internal Registries and Counters
Name Description

aggregateFunctions

Aggregate functions

Used when…​FIXME

allImperativeAggregateFunctions

ImperativeAggregate functions

Used when…​FIXME

allImperativeAggregateFunctionPositions

Positions

Used when…​FIXME

expressionAggInitialProjection

MutableProjection

Used when…​FIXME

generateOutput

Function used to generate an unsafe row (i.e. (UnsafeRow, InternalRow) ⇒ UnsafeRow)

Used when:

groupingAttributes

Grouping attributes

Used when…​FIXME

groupingProjection

UnsafeProjection

Used when…​FIXME

processRow

(InternalRow, InternalRow) ⇒ Unit

Used when…​FIXME

Creating AggregationIterator Instance

AggregationIterator takes the following when created:

AggregationIterator initializes the internal registries and counters.

Note
AggregationIterator is a Scala abstract class and cannot be created directly. It is created indirectly for the concrete AggregationIterators.

initializeAggregateFunctions Internal Method

initializeAggregateFunctions…​FIXME

Note
initializeAggregateFunctions is used when…​FIXME

generateProcessRow Internal Method

generateProcessRow…​FIXME

Note
generateProcessRow is used when…​FIXME

generateResultProjection Method

generateResultProjection…​FIXME

Note

generateResultProjection is used when:

赞(0) 打赏
未经允许不得转载:spark技术分享 » AggregationIterator — Generic Iterator of UnsafeRows for Aggregate Physical Operators
分享到: 更多 (0)

关注公众号:spark技术分享

联系我们联系我们

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

微信扫一扫打赏