spark-sql-spark技术分享-第28页

DeserializeToObject

2012-11-04admin阅读(2183)

DeserializeToObject Unary Logical Operator



case class DeserializeToObject(
  deserializer: Expression,
  outputObjAttr: Attribute,
  child: LogicalPlan) extends UnaryNode with ObjectProducer

case class DeserializeToObject(

deserializer: Expression,

outputObjAttr: Attribute,

child: LogicalPlan) extends UnaryNode with ObjectProducer

DeserializeToObject is a unary logical operator that takes the input row from the input child logical plan and turns it into the input outputObjAttr attribute using the given deserializer expression.

DeserializeToObject is a ObjectProducer which produces domain objects as output. DeserializeToObject‘s output is a single-field safe row containing the produced object.

Note	`DeserializeToObject` is the result of CatalystSerde.deserialize.

DescribeTableCommand

2012-11-03admin阅读(1328)

DescribeTableCommand Logical Command

DescribeTableCommand is a logical command that executes a DESCRIBE TABLE SQL statement.

DescribeTableCommand is created exclusively when SparkSqlAstBuilder is requested to parse DESCRIBE TABLE SQL statement (with no column specified).

DescribeTableCommand uses the following output schema:

col_name as the name of the column
data_type as the data type of the column
comment as the comment of the column



spark.range(1).createOrReplaceTempView("demo")

// DESC view
scala> sql("DESC EXTENDED demo").show
+--------+---------+-------+
|col_name|data_type|comment|
+--------+---------+-------+
|      id|   bigint|   null|
+--------+---------+-------+

// DESC table
// Make the demo reproducible
spark.sharedState.externalCatalog.dropTable(
  db = "default",
  table = "bucketed",
  ignoreIfNotExists = true,
  purge = true)
spark.range(10).write.bucketBy(5, "id").saveAsTable("bucketed")
assert(spark.catalog.tableExists("bucketed"))

// EXTENDED to include Detailed Table Information
// Note no partitions used
// Could also be FORMATTED
scala> sql("DESC EXTENDED bucketed").show(numRows = 50, truncate = false)
+----------------------------+-----------------------------------------------------------------------------+-------+
|col_name                    |data_type                                                                    |comment|
+----------------------------+-----------------------------------------------------------------------------+-------+
|id                          |bigint                                                                       |null   |
|                            |                                                                             |       |
|# Detailed Table Information|                                                                             |       |
|Database                    |default                                                                      |       |
|Table                       |bucketed                                                                     |       |
|Owner                       |jacek                                                                        |       |
|Created Time                |Sun Sep 30 20:57:22 CEST 2018                                                |       |
|Last Access                 |Thu Jan 01 01:00:00 CET 1970                                                 |       |
|Created By                  |Spark 2.3.1                                                                  |       |
|Type                        |MANAGED                                                                      |       |
|Provider                    |parquet                                                                      |       |
|Num Buckets                 |5                                                                            |       |
|Bucket Columns              |[`id`]                                                                       |       |
|Sort Columns                |[]                                                                           |       |
|Table Properties            |[transient_lastDdlTime=1538333842]                                           |       |
|Statistics                  |3740 bytes                                                                   |       |
|Location                    |file:/Users/jacek/dev/apps/spark-2.3.1-bin-hadoop2.7/spark-warehouse/bucketed|       |
|Serde Library               |org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe                           |       |
|InputFormat                 |org.apache.hadoop.mapred.SequenceFileInputFormat                             |       |
|OutputFormat                |org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat                    |       |
|Storage Properties          |[serialization.format=1]                                                     |       |
+----------------------------+-----------------------------------------------------------------------------+-------+

// Make the demo reproducible
val tableName = "partitioned_bucketed_sorted"
val partCol = "part"
spark.sharedState.externalCatalog.dropTable(
  db = "default",
  table = tableName,
  ignoreIfNotExists = true,
  purge = true)
spark.range(10)
  .withColumn("part", $"id" % 2) // extra column for partitions
  .write
  .partitionBy(partCol)
  .bucketBy(5, "id")
  .sortBy("id")
  .saveAsTable(tableName)
assert(spark.catalog.tableExists(tableName))
scala> sql(s"DESC EXTENDED $tableName").show(numRows = 50, truncate = false)
+----------------------------+------------------------------------------------------------------------------------------------+-------+
|col_name                    |data_type                                                                                       |comment|
+----------------------------+------------------------------------------------------------------------------------------------+-------+
|id                          |bigint                                                                                          |null   |
|part                        |bigint                                                                                          |null   |
|# Partition Information     |                                                                                                |       |
|# col_name                  |data_type                                                                                       |comment|
|part                        |bigint                                                                                          |null   |
|                            |                                                                                                |       |
|# Detailed Table Information|                                                                                                |       |
|Database                    |default                                                                                         |       |
|Table                       |partitioned_bucketed_sorted                                                                     |       |
|Owner                       |jacek                                                                                           |       |
|Created Time                |Mon Oct 01 10:05:32 CEST 2018                                                                   |       |
|Last Access                 |Thu Jan 01 01:00:00 CET 1970                                                                    |       |
|Created By                  |Spark 2.3.1                                                                                     |       |
|Type                        |MANAGED                                                                                         |       |
|Provider                    |parquet                                                                                         |       |
|Num Buckets                 |5                                                                                               |       |
|Bucket Columns              |[`id`]                                                                                          |       |
|Sort Columns                |[`id`]                                                                                          |       |
|Table Properties            |[transient_lastDdlTime=1538381132]                                                              |       |
|Location                    |file:/Users/jacek/dev/apps/spark-2.3.1-bin-hadoop2.7/spark-warehouse/partitioned_bucketed_sorted|       |
|Serde Library               |org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe                                              |       |
|InputFormat                 |org.apache.hadoop.mapred.SequenceFileInputFormat                                                |       |
|OutputFormat                |org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat                                       |       |
|Storage Properties          |[serialization.format=1]                                                                        |       |
|Partition Provider          |Catalog                                                                                         |       |
+----------------------------+------------------------------------------------------------------------------------------------+-------+

scala> sql(s"DESCRIBE EXTENDED $tableName PARTITION ($partCol=1)").show(numRows = 50, truncate = false)
+--------------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------+
|col_name                        |data_type                                                                                                                      |comment|
+--------------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------+
|id                              |bigint                                                                                                                         |null   |
|part                            |bigint                                                                                                                         |null   |
|# Partition Information         |                                                                                                                               |       |
|# col_name                      |data_type                                                                                                                      |comment|
|part                            |bigint                                                                                                                         |null   |
|                                |                                                                                                                               |       |
|# Detailed Partition Information|                                                                                                                               |       |
|Database                        |default                                                                                                                        |       |
|Table                           |partitioned_bucketed_sorted                                                                                                    |       |
|Partition Values                |[part=1]                                                                                                                       |       |
|Location                        |file:/Users/jacek/dev/apps/spark-2.3.1-bin-hadoop2.7/spark-warehouse/partitioned_bucketed_sorted/part=1                        |       |
|Serde Library                   |org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe                                                                             |       |
|InputFormat                     |org.apache.hadoop.mapred.SequenceFileInputFormat                                                                               |       |
|OutputFormat                    |org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat                                                                      |       |
|Storage Properties              |[path=file:/Users/jacek/dev/apps/spark-2.3.1-bin-hadoop2.7/spark-warehouse/partitioned_bucketed_sorted, serialization.format=1]|       |
|Partition Parameters            |{totalSize=1870, numFiles=5, transient_lastDdlTime=1538381329}                                                                 |       |
|Partition Statistics            |1870 bytes                                                                                                                     |       |
|                                |                                                                                                                               |       |
|# Storage Information           |                                                                                                                               |       |
|Num Buckets                     |5                                                                                                                              |       |
|Bucket Columns                  |[`id`]                                                                                                                         |       |
|Sort Columns                    |[`id`]                                                                                                                         |       |
|Location                        |file:/Users/jacek/dev/apps/spark-2.3.1-bin-hadoop2.7/spark-warehouse/partitioned_bucketed_sorted                               |       |
|Serde Library                   |org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe                                                                             |       |
|InputFormat                     |org.apache.hadoop.mapred.SequenceFileInputFormat                                                                               |       |
|OutputFormat                    |org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat                                                                      |       |
|Storage Properties              |[serialization.format=1]                                                                                                       |       |
+--------------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------+

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

spark.range(1).createOrReplaceTempView("demo")

// DESC view

scala> sql("DESC EXTENDED demo").show

+--------+---------+-------+

|col_name|data_type|comment|

+--------+---------+-------+

| id| bigint| null|

+--------+---------+-------+

// DESC table

// Make the demo reproducible

spark.sharedState.externalCatalog.dropTable(

db = "default",

table = "bucketed",

ignoreIfNotExists = true,

purge = true)

spark.range(10).write.bucketBy(5, "id").saveAsTable("bucketed")

assert(spark.catalog.tableExists("bucketed"))

// EXTENDED to include Detailed Table Information

// Note no partitions used

// Could also be FORMATTED

scala> sql("DESC EXTENDED bucketed").show(numRows = 50, truncate = false)

+----------------------------+-----------------------------------------------------------------------------+-------+

|col_name |data_type |comment|

+----------------------------+-----------------------------------------------------------------------------+-------+

|id |bigint |null |

| | | |

|# Detailed Table Information| | |

|Database |default | |

|Table |bucketed | |

|Owner |jacek | |

|Created Time |Sun Sep 30 20:57:22 CEST 2018 | |

|Last Access |Thu Jan 01 01:00:00 CET 1970 | |

|Created By |Spark 2.3.1 | |

|Type |MANAGED | |

|Provider |parquet | |

|Num Buckets |5 | |

|Bucket Columns |[`id`] | |

|Sort Columns |[] | |

|Table Properties |[transient_lastDdlTime=1538333842] | |

|Statistics |3740 bytes | |

|Location |file:/Users/jacek/dev/apps/spark-2.3.1-bin-hadoop2.7/spark-warehouse/bucketed| |

|Serde Library |org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | |

|InputFormat |org.apache.hadoop.mapred.SequenceFileInputFormat | |

|OutputFormat |org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat | |

|Storage Properties |[serialization.format=1] | |

+----------------------------+-----------------------------------------------------------------------------+-------+

// Make the demo reproducible

val tableName = "partitioned_bucketed_sorted"

val partCol = "part"

spark.sharedState.externalCatalog.dropTable(

db = "default",

table = tableName,

ignoreIfNotExists = true,

purge = true)

spark.range(10)

.withColumn("part", $"id" % 2) // extra column for partitions

.write

.partitionBy(partCol)

.bucketBy(5, "id")

.sortBy("id")

.saveAsTable(tableName)

assert(spark.catalog.tableExists(tableName))

scala> sql(s"DESC EXTENDED $tableName").show(numRows = 50, truncate = false)

+----------------------------+------------------------------------------------------------------------------------------------+-------+

|col_name |data_type |comment|

+----------------------------+------------------------------------------------------------------------------------------------+-------+

|id |bigint |null |

|part |bigint |null |

|# Partition Information | | |

|# col_name |data_type |comment|

|part |bigint |null |

| | | |

|# Detailed Table Information| | |

|Database |default | |

|Table |partitioned_bucketed_sorted | |

|Owner |jacek | |

|Created Time |Mon Oct 01 10:05:32 CEST 2018 | |

|Last Access |Thu Jan 01 01:00:00 CET 1970 | |

|Created By |Spark 2.3.1 | |

|Type |MANAGED | |

|Provider |parquet | |

|Num Buckets |5 | |

|Bucket Columns |[`id`] | |

|Sort Columns |[`id`] | |

|Table Properties |[transient_lastDdlTime=1538381132] | |

|Location |file:/Users/jacek/dev/apps/spark-2.3.1-bin-hadoop2.7/spark-warehouse/partitioned_bucketed_sorted| |

|Serde Library |org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | |

|InputFormat |org.apache.hadoop.mapred.SequenceFileInputFormat | |

|OutputFormat |org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat | |

|Storage Properties |[serialization.format=1] | |

|Partition Provider |Catalog | |

+----------------------------+------------------------------------------------------------------------------------------------+-------+

scala> sql(s"DESCRIBE EXTENDED $tableName PARTITION ($partCol=1)").show(numRows = 50, truncate = false)

+--------------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------+

|col_name |data_type |comment|

+--------------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------+

|id |bigint |null |

|part |bigint |null |

|# Partition Information | | |

|# col_name |data_type |comment|

|part |bigint |null |

| | | |

|# Detailed Partition Information| | |

|Database |default | |

|Table |partitioned_bucketed_sorted | |

|Partition Values |[part=1] | |

|Location |file:/Users/jacek/dev/apps/spark-2.3.1-bin-hadoop2.7/spark-warehouse/partitioned_bucketed_sorted/part=1 | |

|Serde Library |org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | |

|InputFormat |org.apache.hadoop.mapred.SequenceFileInputFormat | |

|OutputFormat |org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat | |

|Storage Properties |[path=file:/Users/jacek/dev/apps/spark-2.3.1-bin-hadoop2.7/spark-warehouse/partitioned_bucketed_sorted, serialization.format=1]| |

|Partition Parameters |{totalSize=1870, numFiles=5, transient_lastDdlTime=1538381329} | |

|Partition Statistics |1870 bytes | |

| | | |

|# Storage Information | | |

|Num Buckets |5 | |

|Bucket Columns |[`id`] | |

|Sort Columns |[`id`] | |

|Location |file:/Users/jacek/dev/apps/spark-2.3.1-bin-hadoop2.7/spark-warehouse/partitioned_bucketed_sorted | |

|Serde Library |org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | |

|InputFormat |org.apache.hadoop.mapred.SequenceFileInputFormat | |

|OutputFormat |org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat | |

|Storage Properties |[serialization.format=1] | |

+--------------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------+

Executing Logical Command — `run` Method



run(sparkSession: SparkSession): Seq[Row]

run(sparkSession: SparkSession): Seq[Row]

Note	`run` is part of the RunnableCommand Contract to execute (run) a logical command.

run uses the SessionCatalog (of the SessionState of the input SparkSession) and branches off per the type of the table to display.

For a temporary view, run requests the SessionCatalog to lookupRelation to access the schema and describeSchema.

For all other table types, run does the following:

Requests the SessionCatalog to retrieve the table metadata from the external catalog (metastore) (as a CatalogTable) and describeSchema (with the schema)
describePartitionInfo
describeDetailedPartitionInfo if the TablePartitionSpec is available or describeFormattedTableInfo when isExtended flag is on

Describing Detailed Partition and Storage Information — `describeFormattedDetailedPartitionInfo` Internal Method



describeFormattedDetailedPartitionInfo(
  tableIdentifier: TableIdentifier,
  table: CatalogTable,
  partition: CatalogTablePartition,
  buffer: ArrayBuffer[Row]): Unit

describeFormattedDetailedPartitionInfo(

tableIdentifier: TableIdentifier,

table: CatalogTable,

partition: CatalogTablePartition,

buffer: ArrayBuffer[Row]): Unit

describeFormattedDetailedPartitionInfo simply adds the following entries (rows) to the input mutable buffer:

A new line
# Detailed Partition Information
Database with the database of the given table
Table with the table of the given tableIdentifier
Partition specification (of the CatalogTablePartition)
A new line
# Storage Information
Bucketing specification of the table (if defined)
Storage specification of the table

Note	`describeFormattedDetailedPartitionInfo` is used exclusively when `DescribeTableCommand` is requested to describeDetailedPartitionInfo with a non-empty partitionSpec and the isExtended flag on.

Describing Detailed Table Information — `describeFormattedTableInfo` Internal Method



describeFormattedTableInfo(table: CatalogTable, buffer: ArrayBuffer[Row]): Unit

describeFormattedTableInfo(table: CatalogTable, buffer: ArrayBuffer[Row]): Unit

describeFormattedTableInfo…FIXME

Note	`describeFormattedTableInfo` is used exclusively when `DescribeTableCommand` is requested to run for a non-temporary table and the isExtended flag on.

`describeDetailedPartitionInfo` Internal Method



describeDetailedPartitionInfo(
  tableIdentifier: TableIdentifier,
  table: CatalogTable,
  partition: CatalogTablePartition,
  buffer: ArrayBuffer[Row]): Unit

describeDetailedPartitionInfo(

tableIdentifier: TableIdentifier,

table: CatalogTable,

partition: CatalogTablePartition,

buffer: ArrayBuffer[Row]): Unit

describeDetailedPartitionInfo…FIXME

Note	`describeDetailedPartitionInfo` is used exclusively when `DescribeTableCommand` is requested to run with a non-empty partitionSpec.

Creating DescribeTableCommand Instance

DescribeTableCommand takes the following when created:

TableIdentifier
TablePartitionSpec
isExtended flag

DescribeTableCommand initializes the internal registries and counters.

`describeSchema` Internal Method



describeSchema(
  schema: StructType,
  buffer: ArrayBuffer[Row],
  header: Boolean): Unit

describeSchema(

schema: StructType,

buffer: ArrayBuffer[Row],

header: Boolean): Unit

describeSchema…FIXME

Note	`describeSchema` is used when…FIXME

Describing Partition Information — `describePartitionInfo` Internal Method



describePartitionInfo(table: CatalogTable, buffer: ArrayBuffer[Row]): Unit

describePartitionInfo(table: CatalogTable, buffer: ArrayBuffer[Row]): Unit

describePartitionInfo…FIXME

Note	`describePartitionInfo` is used when…FIXME

DescribeColumnCommand

2012-11-02admin阅读(1466)

DescribeColumnCommand Logical Command for DESCRIBE TABLE SQL Command with Column

DescribeColumnCommand is a logical command for DESCRIBE TABLE SQL command with a single column only (i.e. no PARTITION specification).



[DESC|DESCRIBE] TABLE? [EXTENDED|FORMATTED] table_name column_name

[DESC|DESCRIBE] TABLE? [EXTENDED|FORMATTED] table_name column_name



// Make the example reproducible
val tableName = "t1"
import org.apache.spark.sql.catalyst.TableIdentifier
val tableId = TableIdentifier(tableName)

val sessionCatalog = spark.sessionState.catalog
sessionCatalog.dropTable(tableId, ignoreIfNotExists = true, purge = true)

val df = Seq((0, 0.0, "zero"), (1, 1.4, "one")).toDF("id", "p1", "p2")
df.write.saveAsTable("t1")

// DescribeColumnCommand represents DESC EXTENDED tableName colName SQL command
val descExtSQL = "DESC EXTENDED t1 p1"
val plan = spark.sql(descExtSQL).queryExecution.logical
import org.apache.spark.sql.execution.command.DescribeColumnCommand
val cmd = plan.asInstanceOf[DescribeColumnCommand]
scala> println(cmd)
DescribeColumnCommand `t1`, [p1], true

scala> spark.sql(descExtSQL).show
+--------------+----------+
|     info_name|info_value|
+--------------+----------+
|      col_name|        p1|
|     data_type|    double|
|       comment|      NULL|
|           min|      NULL|
|           max|      NULL|
|     num_nulls|      NULL|
|distinct_count|      NULL|
|   avg_col_len|      NULL|
|   max_col_len|      NULL|
|     histogram|      NULL|
+--------------+----------+

// Run ANALYZE TABLE...FOR COLUMNS SQL command to compute the column statistics
val allCols = df.columns.mkString(",")
val analyzeTableSQL = s"ANALYZE TABLE $tableName COMPUTE STATISTICS FOR COLUMNS $allCols"
spark.sql(analyzeTableSQL)

scala> spark.sql(descExtSQL).show
+--------------+----------+
|     info_name|info_value|
+--------------+----------+
|      col_name|        p1|
|     data_type|    double|
|       comment|      NULL|
|           min|       0.0|
|           max|       1.4|
|     num_nulls|         0|
|distinct_count|         2|
|   avg_col_len|         8|
|   max_col_len|         8|
|     histogram|      NULL|
+--------------+----------+

// Make the example reproducible

val tableName = "t1"

import org.apache.spark.sql.catalyst.TableIdentifier

val tableId = TableIdentifier(tableName)

val sessionCatalog = spark.sessionState.catalog

sessionCatalog.dropTable(tableId, ignoreIfNotExists = true, purge = true)

val df = Seq((0, 0.0, "zero"), (1, 1.4, "one")).toDF("id", "p1", "p2")

df.write.saveAsTable("t1")

// DescribeColumnCommand represents DESC EXTENDED tableName colName SQL command

val descExtSQL = "DESC EXTENDED t1 p1"

val plan = spark.sql(descExtSQL).queryExecution.logical

import org.apache.spark.sql.execution.command.DescribeColumnCommand

val cmd = plan.asInstanceOf[DescribeColumnCommand]

scala> println(cmd)

DescribeColumnCommand `t1`, [p1], true

scala> spark.sql(descExtSQL).show

+--------------+----------+

| info_name|info_value|

+--------------+----------+

| col_name| p1|

| data_type| double|

| comment| NULL|

| min| NULL|

| max| NULL|

| num_nulls| NULL|

|distinct_count| NULL|

| avg_col_len| NULL|

| max_col_len| NULL|

| histogram| NULL|

+--------------+----------+

// Run ANALYZE TABLE...FOR COLUMNS SQL command to compute the column statistics

val allCols = df.columns.mkString(",")

val analyzeTableSQL = s"ANALYZE TABLE $tableName COMPUTE STATISTICS FOR COLUMNS $allCols"

spark.sql(analyzeTableSQL)

scala> spark.sql(descExtSQL).show

+--------------+----------+

| info_name|info_value|

+--------------+----------+

| col_name| p1|

| data_type| double|

| comment| NULL|

| min| 0.0|

| max| 1.4|

| num_nulls| 0|

|distinct_count| 2|

| avg_col_len| 8|

| max_col_len| 8|

| histogram| NULL|

+--------------+----------+

DescribeColumnCommand defines the output schema with the following columns:

info_name with “name of the column info” comment
info_value with “value of the column info” comment

Note	`DescribeColumnCommand` is described by `describeTable` labeled alternative in `statement` expression in `SqlBase.g4` and parsed using SparkSqlParser.

Executing Logical Command (Describing Column with Optional Statistics) — `run` Method



run(session: SparkSession): Seq[Row]

run(session: SparkSession): Seq[Row]

Note	`run` is part of RunnableCommand Contract to execute (run) a logical command.

run resolves the column name in table and makes sure that it is a “flat” field (i.e. not of a nested data type).

run requests the SessionCatalog for the table metadata.

Note	`run` uses the input `SparkSession` to access SessionState that in turn is used to access the SessionCatalog.

run takes the column statistics from the table statistics if available.

Note	Column statistics are available (in the table statistics) only after ANALYZE TABLE FOR COLUMNS SQL command was run.

run adds comment metadata if available for the column.

run gives the following rows (in that order):

col_name
data_type
comment

If DescribeColumnCommand command was executed with EXTENDED or FORMATTED option, run gives the following additional rows (in that order):

min
max
num_nulls
distinct_count
avg_col_len
max_col_len
histogram

run gives NULL for the value of the comment and statistics if not available.

`histogramDescription` Internal Method



histogramDescription(histogram: Histogram): Seq[Row]

histogramDescription(histogram: Histogram): Seq[Row]

histogramDescription…FIXME

Note	`histogramDescription` is used exclusively when `DescribeColumnCommand` is executed with `EXTENDED` or `FORMATTED` option turned on.

Creating DescribeColumnCommand Instance

DescribeColumnCommand takes the following when created:

TableIdentifier
Column name
isExtended flag that indicates whether EXTENDED or FORMATTED option was used or not

DataSourceV2Relation

2012-11-01admin阅读(1680)

DataSourceV2Relation Leaf Logical Operator

DataSourceV2Relation is…FIXME

CreateViewCommand

2012-10-31admin阅读(1445)

CreateViewCommand Logical Command

CreateViewCommand is a logical command for creating or replacing a view or a table.

CreateViewCommand is created to represent the following:

CREATE VIEW AS SQL statements
Dataset operators: Dataset.createTempView, Dataset.createOrReplaceTempView, Dataset.createGlobalTempView and Dataset.createOrReplaceGlobalTempView

Caution

FIXME What’s the difference between CreateTempViewUsing?

CreateViewCommand works with different view types.

Table 1. CreateViewCommand Behaviour Per View Type
View Type	Description / Side Effect
`LocalTempView`	A session-scoped local temporary view that is available until the session, that has created it, is stopped. When executed, `CreateViewCommand` requests the current `SessionCatalog` to create a temporary view.
`GlobalTempView`	A cross-session global temporary view that is available until the Spark application stops. When executed, `CreateViewCommand` requests the current `SessionCatalog` to create a global view.
`PersistedView`	A cross-session persisted view that is available until dropped. When executed, `CreateViewCommand` checks if the table exists. If it does and replace is enabled `CreateViewCommand` requests the current `SessionCatalog` to alter a table. Otherwise, when the table does not exist, `CreateViewCommand` requests the current `SessionCatalog` to create it.



/* CREATE [OR REPLACE] [[GLOBAL] TEMPORARY]
VIEW [IF NOT EXISTS] tableIdentifier
[identifierCommentList] [COMMENT STRING]
[PARTITIONED ON identifierList]
[TBLPROPERTIES tablePropertyList] AS query */

// Demo table for "AS query" part
spark.range(10).write.mode("overwrite").saveAsTable("t1")

// The "AS" query
val asQuery = "SELECT * FROM t1"

// The following queries should all work fine
val q1 = "CREATE VIEW v1 AS " + asQuery
sql(q1)

val q2 = "CREATE OR REPLACE VIEW v1 AS " + asQuery
sql(q2)

val q3 = "CREATE OR REPLACE TEMPORARY VIEW v1 " + asQuery
sql(q3)

val q4 = "CREATE OR REPLACE GLOBAL TEMPORARY VIEW v1 " + asQuery
sql(q4)

val q5 = "CREATE VIEW IF NOT EXISTS v1 AS " + asQuery
sql(q5)

// The following queries should all fail
// the number of user-specified columns does not match the schema of the AS query
val qf1 = "CREATE VIEW v1 (c1 COMMENT 'comment', c2) AS " + asQuery
scala> sql(qf1)
org.apache.spark.sql.AnalysisException: The number of columns produced by the SELECT clause (num: `1`) does not match the number of column names specified by CREATE VIEW (num: `2`).;
  at org.apache.spark.sql.execution.command.CreateViewCommand.run(views.scala:134)
  at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
  at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
  at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
  at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:190)
  at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:190)
  at org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3254)
  at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:77)
  at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3253)
  at org.apache.spark.sql.Dataset.<init>(Dataset.scala:190)
  at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:75)
  at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:641)
  ... 49 elided

// CREATE VIEW ... PARTITIONED ON is not allowed
val qf2 = "CREATE VIEW v1 PARTITIONED ON (c1, c2) AS " + asQuery
scala> sql(qf2)
org.apache.spark.sql.catalyst.parser.ParseException:
Operation not allowed: CREATE VIEW ... PARTITIONED ON(line 1, pos 0)

// Use the same name of t1 for a new view
val qf3 = "CREATE VIEW t1 AS " + asQuery
scala> sql(qf3)
org.apache.spark.sql.AnalysisException: `t1` is not a view;
  at org.apache.spark.sql.execution.command.CreateViewCommand.run(views.scala:156)
  at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
  at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
  at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
  at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:190)
  at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:190)
  at org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3254)
  at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:77)
  at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3253)
  at org.apache.spark.sql.Dataset.<init>(Dataset.scala:190)
  at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:75)
  at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:641)
  ... 49 elided

// View already exists
val qf4 = "CREATE VIEW v1 AS " + asQuery
scala> sql(qf4)
org.apache.spark.sql.AnalysisException: View `v1` already exists. If you want to update the view definition, please use ALTER VIEW AS or CREATE OR REPLACE VIEW AS;
  at org.apache.spark.sql.execution.command.CreateViewCommand.run(views.scala:169)
  at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
  at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
  at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
  at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:190)
  at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:190)
  at org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3254)
  at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:77)
  at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3253)
  at org.apache.spark.sql.Dataset.<init>(Dataset.scala:190)
  at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:75)
  at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:641)
  ... 49 elided

/* CREATE [OR REPLACE] [[GLOBAL] TEMPORARY]

VIEW [IF NOT EXISTS] tableIdentifier

[identifierCommentList] [COMMENT STRING]

[PARTITIONED ON identifierList]

[TBLPROPERTIES tablePropertyList] AS query */

// Demo table for "AS query" part

spark.range(10).write.mode("overwrite").saveAsTable("t1")

// The "AS" query

val asQuery = "SELECT * FROM t1"

// The following queries should all work fine

val q1 = "CREATE VIEW v1 AS " + asQuery

sql(q1)

val q2 = "CREATE OR REPLACE VIEW v1 AS " + asQuery

sql(q2)

val q3 = "CREATE OR REPLACE TEMPORARY VIEW v1 " + asQuery

sql(q3)

val q4 = "CREATE OR REPLACE GLOBAL TEMPORARY VIEW v1 " + asQuery

sql(q4)

val q5 = "CREATE VIEW IF NOT EXISTS v1 AS " + asQuery

sql(q5)

// The following queries should all fail

// the number of user-specified columns does not match the schema of the AS query

val qf1 = "CREATE VIEW v1 (c1 COMMENT 'comment', c2) AS " + asQuery

scala> sql(qf1)

org.apache.spark.sql.AnalysisException: The number of columns produced by the SELECT clause (num: `1`) does not match the number of column names specified by CREATE VIEW (num: `2`).;

at org.apache.spark.sql.execution.command.CreateViewCommand.run(views.scala:134)

at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)

at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)

at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)

at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:190)

at org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3254)

at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:77)

at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3253)

at org.apache.spark.sql.Dataset.<init>(Dataset.scala:190)

at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:75)

at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:641)

... 49 elided

// CREATE VIEW ... PARTITIONED ON is not allowed

val qf2 = "CREATE VIEW v1 PARTITIONED ON (c1, c2) AS " + asQuery

scala> sql(qf2)

org.apache.spark.sql.catalyst.parser.ParseException:

Operation not allowed: CREATE VIEW ... PARTITIONED ON(line 1, pos 0)

// Use the same name of t1 for a new view

val qf3 = "CREATE VIEW t1 AS " + asQuery

scala> sql(qf3)

org.apache.spark.sql.AnalysisException: `t1` is not a view;

at org.apache.spark.sql.execution.command.CreateViewCommand.run(views.scala:156)

at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)

at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)

at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)

at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:190)

at org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3254)

at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:77)

at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3253)

at org.apache.spark.sql.Dataset.<init>(Dataset.scala:190)

at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:75)

at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:641)

... 49 elided

// View already exists

val qf4 = "CREATE VIEW v1 AS " + asQuery

scala> sql(qf4)

org.apache.spark.sql.AnalysisException: View `v1` already exists. If you want to update the view definition, please use ALTER VIEW AS or CREATE OR REPLACE VIEW AS;

at org.apache.spark.sql.execution.command.CreateViewCommand.run(views.scala:169)

at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)

at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)

at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)

at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:190)

at org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3254)

at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:77)

at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3253)

at org.apache.spark.sql.Dataset.<init>(Dataset.scala:190)

at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:75)

at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:641)

... 49 elided

CreateViewCommand returns the child logical query plan when requested for the inner nodes (that should be shown as an inner nested tree of this node).



val sqlText = "CREATE VIEW v1 AS " + asQuery
val plan = spark.sessionState.sqlParser.parsePlan(sqlText)
scala> println(plan.numberedTreeString)
00 CreateViewCommand `v1`, SELECT * FROM t1, false, false, PersistedView
01    +- 'Project [*]
02       +- 'UnresolvedRelation `t1`

val sqlText = "CREATE VIEW v1 AS " + asQuery

val plan = spark.sessionState.sqlParser.parsePlan(sqlText)

scala> println(plan.numberedTreeString)

00 CreateViewCommand `v1`, SELECT * FROM t1, false, false, PersistedView

01 +- 'Project [*]

02 +- 'UnresolvedRelation `t1`

Creating CatalogTable — `prepareTable` Internal Method



prepareTable(session: SparkSession, analyzedPlan: LogicalPlan): CatalogTable

prepareTable(session: SparkSession, analyzedPlan: LogicalPlan): CatalogTable

prepareTable…FIXME

Note	`prepareTable` is used exclusively when `CreateViewCommand` logical command is executed.

Executing Logical Command — `run` Method



run(sparkSession: SparkSession): Seq[Row]

run(sparkSession: SparkSession): Seq[Row]

Note	`run` is part of RunnableCommand Contract to execute (run) a logical command.

run requests the input SparkSession for the SessionState that is in turn requested to execute the child logical plan (which simply creates a QueryExecution).

Note

run uses a common idiom in Spark SQL to make sure that a logical plan can be analyzed, i.e.



val qe = sparkSession.sessionState.executePlan(child)
qe.assertAnalyzed()
val analyzedPlan = qe.analyzed

val qe = sparkSession.sessionState.executePlan(child)

qe.assertAnalyzed()

val analyzedPlan = qe.analyzed

run verifyTemporaryObjectsNotExists.

run requests the input SparkSession for the SessionState that is in turn requested for the SessionCatalog.

run then branches off per the ViewType:

For local temporary views, run alias the analyzed plan and requests the SessionCatalog to create or replace a local temporary view
For global temporary views, run also alias the analyzed plan and requests the SessionCatalog to create or replace a global temporary view
For persisted views, run asks the SessionCatalog whether the table exists or not (given TableIdentifier).
- If the table exists and the allowExisting flag is on, run simply does nothing (and exits)
- If the table exists and the replace flag is on, run requests the SessionCatalog for the table metadata and replaces the table, i.e. run requests the SessionCatalog to drop the table followed by re-creating it (with a new CatalogTable)
- If however the table does not exist, run simply requests the SessionCatalog to create it (with a new CatalogTable)

run throws an AnalysisException for persisted views when they already exist, the allowExisting flag is off and the table type is not a view.



[name] is not a view

[name] is not a view

run throws an AnalysisException for persisted views when they already exist and the allowExisting and replace flags are off.



View [name] already exists. If you want to update the view definition, please use ALTER VIEW AS or CREATE OR REPLACE VIEW AS

View [name] already exists. If you want to update the view definition, please use ALTER VIEW AS or CREATE OR REPLACE VIEW AS

run throws an AnalysisException if the userSpecifiedColumns are defined and their numbers is different from the number of output schema attributes of the analyzed logical plan.



The number of columns produced by the SELECT clause (num: `[output.length]`) does not match the number of column names specified by CREATE VIEW (num: `[userSpecifiedColumns.length]`).

The number of columns produced by the SELECT clause (num: `[output.length]`) does not match the number of column names specified by CREATE VIEW (num: `[userSpecifiedColumns.length]`).

Creating CreateViewCommand Instance

CreateViewCommand takes the following when created:

TableIdentifier
User-defined columns (as Seq[(String, Option[String])])
Optional comment
Properties (as Map[String, String])
Optional DDL statement
Child logical plan
allowExisting flag
replace flag
ViewType

`verifyTemporaryObjectsNotExists` Internal Method



verifyTemporaryObjectsNotExists(sparkSession: SparkSession): Unit

verifyTemporaryObjectsNotExists(sparkSession: SparkSession): Unit

verifyTemporaryObjectsNotExists…FIXME

Note	`verifyTemporaryObjectsNotExists` is used exclusively when `CreateViewCommand` logical command is executed.

`aliasPlan` Internal Method



aliasPlan(session: SparkSession, analyzedPlan: LogicalPlan): LogicalPlan

aliasPlan(session: SparkSession, analyzedPlan: LogicalPlan): LogicalPlan

aliasPlan…FIXME

Note	`aliasPlan` is used when `CreateViewCommand` logical command is executed (and prepareTable).

CreateTempViewUsing

2012-10-30admin阅读(1771)

CreateTempViewUsing Logical Command

CreateTempViewUsing is a logical command for creating or replacing a temporary view (global or not) using a data source.

CreateTempViewUsing is created to represent CREATE TEMPORARY VIEW … USING SQL statements.



val sqlText = s"""
    |CREATE GLOBAL TEMPORARY VIEW myTempCsvView
    |(id LONG, name STRING)
    |USING csv
  """.stripMargin
// Logical commands are executed at analysis
scala> sql(sqlText)
res4: org.apache.spark.sql.DataFrame = []

scala> spark.catalog.listTables(spark.sharedState.globalTempViewManager.database).show
+-------------+-----------+-----------+---------+-----------+
|         name|   database|description|tableType|isTemporary|
+-------------+-----------+-----------+---------+-----------+
|mytempcsvview|global_temp|       null|TEMPORARY|       true|
+-------------+-----------+-----------+---------+-----------+

val sqlText = s"""

|CREATE GLOBAL TEMPORARY VIEW myTempCsvView

|(id LONG, name STRING)

|USING csv

""".stripMargin

// Logical commands are executed at analysis

scala> sql(sqlText)

res4: org.apache.spark.sql.DataFrame = []

scala> spark.catalog.listTables(spark.sharedState.globalTempViewManager.database).show

+-------------+-----------+-----------+---------+-----------+

+-------------+-----------+-----------+---------+-----------+

+-------------+-----------+-----------+---------+-----------+

Executing Logical Command — `run` Method



run(sparkSession: SparkSession): Seq[Row]

run(sparkSession: SparkSession): Seq[Row]

Note	`run` is part of RunnableCommand Contract to execute (run) a logical command.

run creates a DataSource and requests it to resolve itself (i.e. create a BaseRelation).

run then requests the input SparkSession to create a DataFrame from the BaseRelation that is used to get the analyzed logical plan (that is the view definition of the temporary table).

Depending on the global flag, run requests the SessionCatalog to createGlobalTempView (global flag is on) or createTempView (global flag is off).

run throws an AnalysisException when executed with hive provider.



Hive data source can only be used with tables, you can't use it with CREATE TEMP VIEW USING

Hive data source can only be used with tables, you can't use it with CREATE TEMP VIEW USING

Creating CreateTempViewUsing Instance

CreateTempViewUsing takes the following when created:

TableIdentifier
Optional user-defined schema (as StructType)
replace flag
global flag
Name of the data source provider
Options (as Map[String, String])

`argString` Method



argString: String

argString: String

Note	`argString` is part of the TreeNode Contract to…FIXME.

argString…FIXME

CreateTableCommand

2012-10-29admin阅读(2391)

CreateTableCommand Logical Command

CreateTableCommand is a logical command that FIXME.

Executing Logical Command — `run` Method



run(session: SparkSession): Seq[Row]

run(session: SparkSession): Seq[Row]

Note	`run` is part of RunnableCommand Contract to execute (run) a logical command.

run…FIXME

CreateTable

2012-10-28admin阅读(2943)

CreateTable Logical Operator

CreateTable is a logical operator that represents (is created for) the following:

DataFrameWriter is requested to createTable (when requested to saveAsTable)
SparkSqlAstBuilder is requested to visitCreateTable (for CREATE TABLE SQL command) or visitCreateHiveTable (for CREATE EXTERNAL TABLE SQL command)
CatalogImpl is requested to create a table

CreateTable requires that the table provider of the CatalogTable is defined or throws an AssertionError:



assertion failed: The table to be created must have a provider.

assertion failed: The table to be created must have a provider.

CreateTable can never be resolved and is replaced (resolved) with a logical command at analysis phase in the following rules:

(for non-hive data source tables) DataSourceAnalysis posthoc logical resolution rule to a CreateDataSourceTableCommand or a CreateDataSourceTableAsSelectCommand logical command (when the query was defined or not, respectively)
(for hive tables) HiveAnalysis post-hoc logical resolution rule to a CreateTableCommand or a CreateHiveTableAsSelectCommand logical command (when query was defined or not, respectively)

Creating CreateTable Instance

CreateTable takes the following when created:

CreateTable initializes the internal registries and counters.

CreateHiveTableAsSelectCommand

2012-10-27admin阅读(1218)

CreateHiveTableAsSelectCommand Logical Command

CreateHiveTableAsSelectCommand is a logical command that FIXME.

Executing Logical Command — `run` Method



run(session: SparkSession): Seq[Row]

run(session: SparkSession): Seq[Row]

Note	`run` is part of RunnableCommand Contract to execute (run) a logical command.

run…FIXME

CreateDataSourceTableCommand

2012-10-26admin阅读(1411)

CreateDataSourceTableCommand Logical Command

CreateDataSourceTableCommand is a logical command that creates a new table (in a session-scoped SessionCatalog).

CreateDataSourceTableCommand is created exclusively when DataSourceAnalysis posthoc logical resolution rule resolves a CreateTable logical operator for a non-Hive table provider with no query.

CreateDataSourceTableCommand takes a table metadata and ignoreIfExists flag.

Executing Logical Command — `run` Method



run(sparkSession: SparkSession): Seq[Row]

run(sparkSession: SparkSession): Seq[Row]

Note	`run` is part of RunnableCommand Contract to execute (run) a logical command.

run creates a new table in a session-scoped SessionCatalog.

Note	`run` uses the input `SparkSession` to access SessionState that in turn is used to access the current SessionCatalog.

Internally, run creates a BaseRelation to access the table’s schema.

Caution

FIXME

Note	`run` accepts tables only (not views) with the provider defined.

上一页
1
···
25
26
27
28
29
30
31
...
下一页
共 58 页

spark-sql 第28页

DeserializeToObject Unary Logical Operator

DescribeTableCommand Logical Command

Executing Logical Command — run Method

Describing Detailed Partition and Storage Information — describeFormattedDetailedPartitionInfo Internal Method

Describing Detailed Table Information — describeFormattedTableInfo Internal Method

describeDetailedPartitionInfo Internal Method

Creating DescribeTableCommand Instance

describeSchema Internal Method

Describing Partition Information — describePartitionInfo Internal Method

DescribeColumnCommand Logical Command for DESCRIBE TABLE SQL Command with Column

Executing Logical Command (Describing Column with Optional Statistics) — run Method

histogramDescription Internal Method

Creating DescribeColumnCommand Instance

DataSourceV2Relation Leaf Logical Operator

CreateViewCommand Logical Command

Creating CatalogTable — prepareTable Internal Method

Executing Logical Command — run Method

Creating CreateViewCommand Instance

verifyTemporaryObjectsNotExists Internal Method

aliasPlan Internal Method

CreateTempViewUsing Logical Command

Executing Logical Command — run Method

Creating CreateTempViewUsing Instance

argString Method

CreateTableCommand Logical Command

Executing Logical Command — run Method

CreateTable Logical Operator

Creating CreateTable Instance

CreateHiveTableAsSelectCommand Logical Command

Executing Logical Command — run Method

CreateDataSourceTableCommand Logical Command

Executing Logical Command — run Method

欢迎关注：spark技术分享

关注公众号：spark技术分享

QQ咨询

回顶部

Executing Logical Command — `run` Method

Describing Detailed Partition and Storage Information — `describeFormattedDetailedPartitionInfo` Internal Method

Describing Detailed Table Information — `describeFormattedTableInfo` Internal Method

`describeDetailedPartitionInfo` Internal Method

`describeSchema` Internal Method

Describing Partition Information — `describePartitionInfo` Internal Method

Executing Logical Command (Describing Column with Optional Statistics) — `run` Method

`histogramDescription` Internal Method

Creating CatalogTable — `prepareTable` Internal Method

Executing Logical Command — `run` Method

`verifyTemporaryObjectsNotExists` Internal Method

`aliasPlan` Internal Method

Executing Logical Command — `run` Method

`argString` Method

Executing Logical Command — `run` Method

Executing Logical Command — `run` Method

Executing Logical Command — `run` Method