spark技术分享HiveTableScanExec Leaf Physical Operator
HiveTableScanExec is a leaf physical operator that represents a HiveTableRelation logical operator at execution time.
HiveTableScanExec is created exclusively when HiveTableScans execution planning strategy plans a HiveTableRelation logical operator (i.e. is executed on a logical query plan with a HiveTableRelation logical operator).
| Key | Name (in web UI) | Description |
|---|---|---|
|
|
number of output rows |
| Name | Description |
|---|---|
|
|
Hive’s Used when |
|
|
|
|
|
Creating HiveTableScanExec Instance
HiveTableScanExec takes the following when created:
-
Requested attributes
-
Partition pruning predicate expression
HiveTableScanExec initializes the internal registries and counters.
Executing Physical Operator (Generating RDD[InternalRow]) — doExecute Method
|
1 2 3 4 5 |
doExecute(): RDD[InternalRow] |
|
Note
|
doExecute is part of SparkPlan Contract to generate the runtime representation of a structured query as a distributed computation over internal binary rows on Apache Spark (i.e. RDD[InternalRow]).
|
doExecute…FIXME
