spark技术分享ReplaceExpressions Logical Optimization
ReplaceExpressions is a base logical optimization that replaces RuntimeReplaceable expressions with their single child expression.
ReplaceExpressions is part of the Finish Analysis once-executed batch in the standard batches of the Catalyst Optimizer.
ReplaceExpressions is simply a Catalyst rule for transforming logical plans, i.e. Rule[LogicalPlan].
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
val query = sql("select ifnull(NULL, array('2')) from values 1") val analyzedPlan = query.queryExecution.analyzed scala> println(analyzedPlan.numberedTreeString) 00 Project [ifnull(null, array(2)) AS ifnull(NULL, array('2'))#3] 01 +- LocalRelation [col1#2] import org.apache.spark.sql.catalyst.optimizer.ReplaceExpressions val optimizedPlan = ReplaceExpressions(analyzedPlan) scala> println(optimizedPlan.numberedTreeString) 00 Project [coalesce(cast(null as array<string>), cast(array(2) as array<string>)) AS ifnull(NULL, array('2'))#3] 01 +- LocalRelation [col1#2] |
Executing Rule — apply Method
|
1 2 3 4 5 |
apply(plan: LogicalPlan): LogicalPlan |
|
Note
|
apply is part of the Rule Contract to execute (apply) a rule on a TreeNode (e.g. LogicalPlan).
|
apply traverses all Catalyst expressions (in the input LogicalPlan) and replaces a RuntimeReplaceable expression into its single child.
