spark技术分享CreateTable Logical Operator
CreateTable is a logical operator that represents (is created for) the following:
-
DataFrameWriteris requested to createTable (when requested to saveAsTable) -
SparkSqlAstBuilderis requested to visitCreateTable (forCREATE TABLESQL command) or visitCreateHiveTable (forCREATE EXTERNAL TABLESQL command) -
CatalogImplis requested to create a table
CreateTable requires that the table provider of the CatalogTable is defined or throws an AssertionError:
|
1 2 3 4 5 |
assertion failed: The table to be created must have a provider. |
CreateTable can never be resolved and is replaced (resolved) with a logical command at analysis phase in the following rules:
-
(for non-hive data source tables) DataSourceAnalysis posthoc logical resolution rule to a CreateDataSourceTableCommand or a CreateDataSourceTableAsSelectCommand logical command (when the query was defined or not, respectively)
-
(for hive tables) HiveAnalysis post-hoc logical resolution rule to a CreateTableCommand or a CreateHiveTableAsSelectCommand logical command (when query was defined or not, respectively)
Creating CreateTable Instance
CreateTable takes the following when created:
CreateTable initializes the internal registries and counters.
