spark技术分享AliasViewChild Logical Analysis Rule
AliasViewChild is a logical analysis rule that transforms a logical query plan with View unary logical operators and adds Project logical operator (possibly with Alias expressions) when the outputs of a view and the underlying table do not match (and therefore require aliasing and projection).
AliasViewChild is simply a Catalyst rule for transforming logical plans, i.e. Rule[LogicalPlan].
AliasViewChild takes a SQLConf when created.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
// Sample view for the demo // The order of the column names do not match // In View: name, id // In VALUES: id, name sql(""" CREATE OR REPLACE VIEW v (name COMMENT 'First name only', id COMMENT 'Identifier') COMMENT 'Permanent view' AS VALUES (1, 'Jacek'), (2, 'Agata') AS t1(id, name) """) scala> :type spark org.apache.spark.sql.SparkSession val q = spark.table("v") val plan = q.queryExecution.logical scala> println(plan.numberedTreeString) 00 'UnresolvedRelation `v` // Resolve UnresolvedRelation first // since AliasViewChild work with View operators only import spark.sessionState.analyzer.ResolveRelations val resolvedPlan = ResolveRelations(plan) scala> println(resolvedPlan.numberedTreeString) 00 SubqueryAlias v 01 +- View (`default`.`v`, [name#32,id#33]) 02 +- SubqueryAlias t1 03 +- LocalRelation [id#34, name#35] scala> :type spark.sessionState.conf org.apache.spark.sql.internal.SQLConf import org.apache.spark.sql.catalyst.analysis.AliasViewChild val rule = AliasViewChild(spark.sessionState.conf) // Notice that resolvedPlan is used (not plan) val planAfterAliasViewChild = rule(resolvedPlan) scala> println(planAfterAliasViewChild.numberedTreeString) 00 SubqueryAlias v 01 +- View (`default`.`v`, [name#32,id#33]) 02 +- Project [cast(id#34 as int) AS name#32, cast(name#35 as string) AS id#33] 03 +- SubqueryAlias t1 04 +- LocalRelation [id#34, name#35] |
Executing Rule — apply Method
|
1 2 3 4 5 |
apply(plan: LogicalPlan): LogicalPlan |
|
Note
|
apply is part of the Rule Contract to execute (apply) a rule on a TreeNode (e.g. LogicalPlan).
|
apply…FIXME
