RowDataSourceScanExec Leaf Physical Operator
RowDataSourceScanExec
is a DataSourceScanExec (and so indirectly a leaf physical operator) for scanning data from a BaseRelation.
RowDataSourceScanExec
is created to represent a LogicalRelation with the following scan types when DataSourceStrategy
execution planning strategy is executed:
-
CatalystScan
,PrunedFilteredScan
,PrunedScan
(indirectly whenDataSourceStrategy
is requested to pruneFilterProjectRaw) -
TableScan
RowDataSourceScanExec
marks the filters that are included in the handledFilters with *
(star) in the metadata that is used for a simple text representation.
1 2 3 4 5 |
// DEMO RowDataSourceScanExec with a simple text representation with stars |
Generating Java Source Code for Produce Path in Whole-Stage Code Generation — doProduce
Method
1 2 3 4 5 |
doProduce(ctx: CodegenContext): String |
Note
|
doProduce is part of CodegenSupport Contract to generate the Java source code for produce path in Whole-Stage Code Generation.
|
doProduce
…FIXME
Creating RowDataSourceScanExec Instance
RowDataSourceScanExec
takes the following when created:
-
Output schema attributes
-
Handled filter predicates
-
RDD of internal binary rows
Note
|
The input filter predicates and handled filters predicates are used exclusively for the metadata property that is part of DataSourceScanExec Contract to describe a scan for a simple text representation (in a query plan tree). |
metadata
Property
1 2 3 4 5 |
metadata: Map[String, String] |
Note
|
metadata is part of DataSourceScanExec Contract to describe a scan for a simple text representation (in a query plan tree).
|
metadata
marks the filter predicates that are included in the handled filters predicates with *
(star).
Note
|
Filter predicates with * (star) are to denote filters that are pushed down to a relation (aka data source).
|
In the end, metadata
creates the following mapping:
-
ReadSchema with the output converted to catalog representation
-
PushedFilters with the marked and unmarked filter predicates