spark技术分享Expand Unary Logical Operator
Expand is a unary logical operator that represents Cube, Rollup, GroupingSets and TimeWindow logical operators after they have been resolved at analysis phase.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
FIXME Examples for 1. Cube 2. Rollup 3. GroupingSets 4. See TimeWindow val q = ... scala> println(q.queryExecution.logical.numberedTreeString) ... |
|
Note
|
Expand logical operator is resolved to ExpandExec physical operator in BasicOperators execution planning strategy.
|
| Name | Description |
|---|---|
|
|
|
|
|
Empty set of expressions |
Analysis Phase
Expand logical operator is resolved to at analysis phase in the following logical evaluation rules:
-
ResolveGroupingAnalytics (for
Cube,Rollup, GroupingSets logical operators) -
TimeWindowing (for TimeWindow logical operator)
|
Note
|
Aggregate → (Cube|Rollup|GroupingSets) → constructAggregate → constructExpand |
|
1 2 3 4 5 6 7 8 9 10 |
val spark: SparkSession = ... // using q from the example above val plan = q.queryExecution.logical scala> println(plan.numberedTreeString) ...FIXME |
Creating Expand Instance
Expand takes the following when created:
-
Projection expressions
-
Output schema attributes
-
Child logical plan
