spark技术分享ShowCreateTableCommand Logical Command
ShowCreateTableCommand is a logical command that executes a SHOW CREATE TABLE SQL statement (with a data source / non-Hive or a Hive table).
ShowCreateTableCommand is created when SparkSqlAstBuilder is requested to parse SHOW CREATE TABLE SQL statement.
ShowCreateTableCommand uses a single createtab_stmt column (of type StringType) for the output schema.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
import org.apache.spark.sql.SaveMode spark.range(10e4.toLong) .write .bucketBy(4, "id") .sortBy("id") .mode(SaveMode.Overwrite) .saveAsTable("bucketed_4_10e4") scala> sql("SHOW CREATE TABLE bucketed_4_10e4").show(truncate = false) +----------------------------------------------------------------------------------------------------------------------------------------------------+ |createtab_stmt | +----------------------------------------------------------------------------------------------------------------------------------------------------+ |CREATE TABLE `bucketed_4_10e4` (`id` BIGINT) USING parquet OPTIONS ( `serialization.format` '1' ) CLUSTERED BY (id) SORTED BY (id) INTO 4 BUCKETS | +----------------------------------------------------------------------------------------------------------------------------------------------------+ scala> sql("SHOW CREATE TABLE bucketed_4_10e4").as[String].collect.foreach(println) CREATE TABLE `bucketed_4_10e4` (`id` BIGINT) USING parquet OPTIONS ( `serialization.format` '1' ) CLUSTERED BY (id) SORTED BY (id) INTO 4 BUCKETS |
ShowCreateTableCommand takes a single TableIdentifier when created.
Executing Logical Command — run Method
|
1 2 3 4 5 |
run(sparkSession: SparkSession): Seq[Row] |
|
Note
|
run is part of RunnableCommand Contract to execute (run) a logical command.
|
run requests the SparkSession for the SessionState that is used to access the SessionCatalog.
run then requests the SessionCatalog to retrieve the table metadata from the external catalog (metastore).
run then showCreateDataSourceTable for a data source / non-Hive table or showCreateHiveTable for a Hive table (per the table metadata).
In the end, run returns the CREATE TABLE statement in a single Row.
showHiveTableNonDataColumns Internal Method
|
1 2 3 4 5 |
showHiveTableNonDataColumns(metadata: CatalogTable, builder: StringBuilder): Unit |
showHiveTableNonDataColumns…FIXME
|
Note
|
showHiveTableNonDataColumns is used exclusively when ShowCreateTableCommand logical command is requested to showCreateHiveTable.
|
showCreateHiveTable Internal Method
|
1 2 3 4 5 |
showCreateHiveTable(metadata: CatalogTable): String |
showCreateHiveTable…FIXME
|
Note
|
showCreateHiveTable is used exclusively when ShowCreateTableCommand logical command is executed (with a Hive table).
|
showHiveTableHeader Internal Method
|
1 2 3 4 5 |
showHiveTableHeader(metadata: CatalogTable, builder: StringBuilder): Unit |
showHiveTableHeader…FIXME
|
Note
|
showHiveTableHeader is used exclusively when ShowCreateTableCommand logical command is requested to showCreateHiveTable.
|
