spark技术分享TypedAggregateExpression Expression
TypedAggregateExpression is the contract for AggregateFunction expressions that…FIXME
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
package org.apache.spark.sql.execution.aggregate trait TypedAggregateExpression extends AggregateFunction { // only required methods that have no implementation def aggregator: Aggregator[Any, Any, Any] def inputClass: Option[Class[_]] def inputDeserializer: Option[Expression] def inputSchema: Option[StructType] def withInputInfo(deser: Expression, cls: Class[_], schema: StructType): TypedAggregateExpression } |
TypedAggregateExpression is used when:
-
TypedColumnis requested to withInputType (for Dataset.select, KeyValueGroupedDataset.agg and RelationalGroupedDataset.agg operators) -
Columnis requested to generateAlias and named (for Dataset.select and KeyValueGroupedDataset.agg operators) -
RelationalGroupedDatasetis requested to alias (whenRelationalGroupedDatasetis requested to create a DataFrame from aggregate expressions)
| Method | Description |
|---|---|
|
Used when…FIXME |
|
|
Used when…FIXME |
|
|
Used when…FIXME |
|
|
Used when…FIXME |
| Aggregator | Description |
|---|---|
Creating TypedAggregateExpression — apply Factory Method
|
1 2 3 4 5 6 |
apply[BUF : Encoder, OUT : Encoder]( aggregator: Aggregator[_, BUF, OUT]): TypedAggregateExpression |
apply…FIXME
|
Note
|
apply is used exclusively when Aggregator is requested to convert itself to a TypedColumn.
|
