spark技术分享SubqueryAlias Unary Logical Operator
SubqueryAlias is a unary logical operator that represents an aliased subquery (i.e. the child logical query plan with the alias in the output schema).
SubqueryAlias is created when:
-
AstBuilderis requested to parse a named or aliased query, aliased query plan and mayApplyAliasPlan in a SQL statement -
Dataset.as operator is used
-
SessionCatalogis requested to find a table or view in catalogs -
RewriteCorrelatedScalarSubquerylogical optimization is requested to constructLeftJoins (when applied to Aggregate, Project or Filter logical operators with correlated scalar subqueries)
SubqueryAlias simply requests the child logical operator for the canonicalized version.
When requested for output schema attributes, SubqueryAlias requests the child logical operator for them and adds the alias as a qualifier.
|
Note
|
EliminateSubqueryAliases logical optimization eliminates (removes) SubqueryAlias operators from a logical query plan.
|
|
Note
|
RewriteCorrelatedScalarSubquery logical optimization rewrites correlated scalar subqueries with SubqueryAlias operators.
|
Catalyst DSL — subquery And as Operators
|
1 2 3 4 5 6 |
as(alias: String): LogicalPlan subquery(alias: Symbol): LogicalPlan |
subquery and as operators in Catalyst DSL create a SubqueryAlias logical operator, e.g. for testing or Spark SQL internals exploration.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
import org.apache.spark.sql.catalyst.dsl.plans._ val t1 = table("t1") val plan = t1.subquery('a) scala> println(plan.numberedTreeString) 00 'SubqueryAlias a 01 +- 'UnresolvedRelation `t1` val plan = t1.as("a") scala> println(plan.numberedTreeString) 00 'SubqueryAlias a 01 +- 'UnresolvedRelation `t1` |
