spark技术分享DataSourceAnalysis PostHoc Logical Resolution Rule
DataSourceAnalysis is a posthoc logical resolution rule that the default and Hive-specific logical query plan analyzers use to FIXME.
| Source Operator | Target Operator | Description |
|---|---|---|
|
CreateTable (isDatasourceTable + no query) |
||
|
CreateTable (isDatasourceTable + a resolved query) |
||
|
InsertIntoDir (non-hive provider) |
||
Technically, DataSourceAnalysis is a Catalyst rule for transforming logical plans, i.e. Rule[LogicalPlan].
|
1 2 3 4 5 6 7 8 9 10 11 |
// FIXME Example of DataSourceAnalysis import org.apache.spark.sql.execution.datasources.DataSourceAnalysis val rule = DataSourceAnalysis(spark.sessionState.conf) val plan = FIXME rule(plan) |
Executing Rule — apply Method
|
1 2 3 4 5 |
apply(plan: LogicalPlan): LogicalPlan |
|
Note
|
apply is part of the Rule Contract to execute (apply) a rule on a TreeNode (e.g. LogicalPlan).
|
apply…FIXME
