spark技术分享ResolveOrdinalInOrderByAndGroupBy Logical Resolution Rule
ResolveOrdinalInOrderByAndGroupBy is a logical resolution rule that converts ordinal positions in Sort and Aggregate logical operators with corresponding expressions in a logical query plan.
ResolveOrdinalInOrderByAndGroupBy is part of the Resolution fixed-point batch in the standard batches of the Analyzer.
ResolveOrdinalInOrderByAndGroupBy is simply a Catalyst rule for transforming logical plans, i.e. Rule[LogicalPlan].
ResolveOrdinalInOrderByAndGroupBy takes no arguments when created.
|
1 2 3 4 5 6 7 8 9 10 11 |
// FIXME: DEMO val rule = spark.sessionState.analyzer.ResolveOrdinalInOrderByAndGroupBy val plan = ??? val planResolved = rule(plan) scala> println(planResolved.numberedTreeString) 00 'UnresolvedRelation `t1` |
Executing Rule — apply Method
|
1 2 3 4 5 |
apply(plan: LogicalPlan): LogicalPlan |
|
Note
|
apply is part of the Rule Contract to execute (apply) a rule on a TreeNode (e.g. LogicalPlan).
|
apply walks the logical plan from children up the tree and looks for Sort and Aggregate logical operators with UnresolvedOrdinal leaf expressions (in ordering and grouping expressions, respectively).
For a Sort logical operator with UnresolvedOrdinal expressions, apply replaces all the SortOrder expressions (with UnresolvedOrdinal child expressions) with SortOrder expressions and the expression at the index - 1 position in the output schema of the child logical operator.
For a Aggregate logical operator with UnresolvedOrdinal expressions, apply replaces all the expressions (with UnresolvedOrdinal child expressions) with the expression at the index - 1 position in the aggregate named expressions of the current Aggregate logical operator.
apply throws a AnalysisException (and hence fails an analysis) if the ordinal is outside the range:
|
1 2 3 4 5 6 |
ORDER BY position [index] is not in select list (valid range is [1, [output.size]]) GROUP BY position [index] is not in select list (valid range is [1, [aggs.size]]) |
