spark技术分享Window Unary Logical Operator
Window is a unary logical operator that…FIXME
Window is created when:
When requested for output schema attributes, Window requests the child logical operator for them and adds the attributes of the window named expressions.
|
Note
|
Window logical operator is a subject of pruning unnecessary window expressions in ColumnPruning logical optimization and collapsing window operators in CollapseWindow logical optimization.
|
|
Note
|
Window logical operator is resolved to a WindowExec in BasicOperators execution planning strategy.
|
Catalyst DSL — window Operator
|
1 2 3 4 5 6 7 8 |
window( windowExpressions: Seq[NamedExpression], partitionSpec: Seq[Expression], orderSpec: Seq[SortOrder]): LogicalPlan |
window operator in Catalyst DSL creates a Window logical operator, e.g. for testing or Spark SQL internals exploration.
|
1 2 3 4 5 |
// FIXME: DEMO |
Creating Window Instance
Window takes the following when created:
-
Window named expressions
-
Window partition specification expressions
-
Window order specification (as a collection of
SortOrderexpressions) -
Child logical operator
Creating AttributeSet with Window Expression Attributes — windowOutputSet Method
|
1 2 3 4 5 |
windowOutputSet: AttributeSet |
windowOutputSet simply creates a AttributeSet with the attributes of the window named expressions.
|
Note
|
|
