CreateStruct Function Builder-spark技术分享

CreateStruct Function Builder

CreateStruct is a function builder (e.g. Seq[Expression] ⇒ Expression) that can create CreateNamedStruct expressions and is the metadata of the struct function.



registryEntry: (String, (ExpressionInfo, FunctionBuilder))

registryEntry: (String, (ExpressionInfo, FunctionBuilder))

registryEntry…FIXME

Note	`registryEntry` is used exclusively when `FunctionRegistry` is requested for the function expression registry.



apply(children: Seq[Expression]): CreateNamedStruct

apply(children: Seq[Expression]): CreateNamedStruct

Note	`apply` is part of Scala’s scala.Function1 contract to create a function of one parameter (e.g. `Seq[Expression]`).

apply creates a CreateNamedStruct expression with the input children expressions as follows:

For NamedExpression expressions that are resolved, apply creates a pair of a Literal expression (with the name of the NamedExpression) and the NamedExpression itself
For NamedExpression expressions that are not resolved yet, apply creates a pair of a NamePlaceholder expression and the NamedExpression itself
For all other expressions, apply creates a pair of a Literal expression (with the value as col[index]) and the Expression itself

Note

apply is used when:

ResolveReferences logical resolution rule is requested to expandStarExpression
InConversion type coercion rule is requested to coerceTypes
ExpressionEncoder is requested to create an ExpressionEncoder for a tuple
Stack generator expression is requested to generate a Java source code
AstBuilder is requested to parse a struct and row constructor
ColumnStat is requested to statExprs
KeyValueGroupedDataset is requested to aggUntyped (when KeyValueGroupedDataset.agg typed operator is used)
Dataset.joinWith typed transformation is used
struct standard function is used
SimpleTypedAggregateExpression expression is requested for the evaluateExpression and resultObjToRow