spark技术分享EstimationUtils
EstimationUtils is…FIXME
getOutputSize Method
|
1 2 3 4 5 6 7 8 |
getOutputSize( attributes: Seq[Attribute], outputRowCount: BigInt, attrStats: AttributeMap[ColumnStat] = AttributeMap(Nil)): BigInt |
getOutputSize…FIXME
|
Note
|
getOutputSize is used when…FIXME
|
nullColumnStat Method
|
1 2 3 4 5 |
nullColumnStat(dataType: DataType, rowCount: BigInt): ColumnStat |
nullColumnStat…FIXME
|
Note
|
nullColumnStat is used exclusively when JoinEstimation is requested to estimateInnerOuterJoin for LeftOuter and RightOuter joins.
|
Checking Availability of Row Count Statistic — rowCountsExist Method
|
1 2 3 4 5 |
rowCountsExist(plans: LogicalPlan*): Boolean |
rowCountsExist is positive (i.e. true) when every logical plan (in the input plans) has estimated number of rows (aka row count) statistic computed.
Otherwise, rowCountsExist is negative (i.e. false).
|
Note
|
rowCountsExist uses LogicalPlanStats to access the estimated statistics and query hints of a logical plan.
|
|
Note
|
|
