关注 spark技术分享,
撸spark源码 玩spark最佳实践

DescribeColumnCommand

DescribeColumnCommand Logical Command for DESCRIBE TABLE SQL Command with Column

DescribeColumnCommand is a logical command for DESCRIBE TABLE SQL command with a single column only (i.e. no PARTITION specification).

DescribeColumnCommand defines the output schema with the following columns:

  • info_name with “name of the column info” comment

  • info_value with “value of the column info” comment

Note
DescribeColumnCommand is described by describeTable labeled alternative in statement expression in SqlBase.g4 and parsed using SparkSqlParser.

Executing Logical Command (Describing Column with Optional Statistics) — run Method

Note
run is part of RunnableCommand Contract to execute (run) a logical command.

run resolves the column name in table and makes sure that it is a “flat” field (i.e. not of a nested data type).

run requests the SessionCatalog for the table metadata.

Note
run uses the input SparkSession to access SessionState that in turn is used to access the SessionCatalog.

run takes the column statistics from the table statistics if available.

Note
Column statistics are available (in the table statistics) only after ANALYZE TABLE FOR COLUMNS SQL command was run.

run adds comment metadata if available for the column.

run gives the following rows (in that order):

  1. col_name

  2. data_type

  3. comment

If DescribeColumnCommand command was executed with EXTENDED or FORMATTED option, run gives the following additional rows (in that order):

  1. min

  2. max

  3. num_nulls

  4. distinct_count

  5. avg_col_len

  6. max_col_len

  7. histogram

run gives NULL for the value of the comment and statistics if not available.

histogramDescription Internal Method

histogramDescription…​FIXME

Note
histogramDescription is used exclusively when DescribeColumnCommand is executed with EXTENDED or FORMATTED option turned on.

Creating DescribeColumnCommand Instance

DescribeColumnCommand takes the following when created:

赞(0) 打赏
未经允许不得转载:spark技术分享 » DescribeColumnCommand
分享到: 更多 (0)

关注公众号:spark技术分享

联系我们联系我们

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

微信扫一扫打赏