spark技术分享Catalyst Rule — Named Transformation of TreeNodes
Rule is a named transformation that can be applied to (i.e. executed on or transform) a TreeNode to produce a new TreeNode.
|
1 2 3 4 5 6 7 8 9 10 11 |
package org.apache.spark.sql.catalyst.rules abstract class Rule[TreeType <: TreeNode[_]] { // only required properties (vals and methods) that have no implementation // the others follow def apply(plan: TreeType): TreeType } |
|
Note
|
TreeType is the type of the TreeNode implementation that a Rule can be applied to, i.e. LogicalPlan, SparkPlan or Expression or a combination thereof.
|
Rule has a rule name (that is the class name of a rule).
|
1 2 3 4 5 |
ruleName: String |
Rule is mainly used to create a batch of rules for a RuleExecutor.
The other notable use cases of Rule are as follows:
-
When
ExperimentalMethodsis requested for extraOptimizations -
When
BaseSessionStateBuilderis requested for customResolutionRules, customPostHocResolutionRules, customOperatorOptimizationRules, and the Optimizer -
When
Analyzeris requested for extendedResolutionRules and postHocResolutionRules (see BaseSessionStateBuilder and HiveSessionStateBuilder) -
When
Optimizeris requested for extendedOperatorOptimizationRules -
When
QueryExecutionis requested for preparations
