spark技术分享LeafNode — Base Logical Operator with No Child Operators and Optional Statistics
LeafNode is the base of logical operators that have no child operators.
LeafNode that wants to survive analysis has to define computeStats as it throws an UnsupportedOperationException by default.
| LeafNode | Description |
|---|---|
Computing Statistics — computeStats Method
|
1 2 3 4 5 |
computeStats(): Statistics |
computeStats simply throws an UnsupportedOperationException.
|
Note
|
Logical operators, e.g. ExternalRDD, LogicalRDD and DataSourceV2Relation, or relations, e.g. HadoopFsRelation or BaseRelation, use spark.sql.defaultSizeInBytes internal property for the default estimated size if the statistics could not be computed.
|
|
Note
|
computeStats is used exclusively when SizeInBytesOnlyStatsPlanVisitor uses the default case to compute the size statistic (in bytes) for a logical operator.
|
