spark技术分享AggregateExpression — Unevaluable Expression Container for AggregateFunction
AggregateExpression is an unevaluable expression (i.e. with no support for eval and doGenCode methods) that acts as a container for an AggregateFunction.
AggregateExpression contains the following:
-
isDistinctflag indicating whether this aggregation is distinct or not (e.g. whether SQL’sDISTINCTkeyword was used for the aggregate function)
AggregateExpression is created when:
-
Analyzerresolves AggregateFunctions (and creates anAggregateExpressionwithCompleteaggregate mode for the functions) -
UserDefinedAggregateFunctionis created withisDistinctflag disabled or enabled -
AggUtilsis requested to planAggregateWithOneDistinct (and createsAggregateExpressionswithPartialandFinalaggregate modes for the functions) -
Aggregatoris requested for a TypedColumn (usingAggregator.toColumn) -
AggregateFunctionis wrapped in a AggregateExpression
| Prefix | AggregateMode |
|---|---|
|
|
|
|
|
|
|
(empty) |
|
| Name | Description |
|---|---|
|
|
AggregateExpression with AggregateFunction expression |
|
|
AggregateFunction expression (for which |
|
|
DataType of AggregateFunction expression |
|
|
Disabled (i.e. |
|
|
Whether or not AggregateFunction expression is nullable. |
|
|
|
|
|
Attribute that is:
|
|
|
Requests AggregateFunction to generate SQL output (with isDistinct flag). |
|
|
Prefix per AggregateMode followed by AggregateFunction‘s |
