spark技术分享BroadcastDistribution
BroadcastDistribution is a Distribution that indicates to use one partition only and…FIXME.
BroadcastDistribution is created when:
-
BroadcastHashJoinExecis requested for required child output distributions (with HashedRelationBroadcastMode of the build join keys) -
BroadcastNestedLoopJoinExecis requested for required child output distributions (with IdentityBroadcastMode)
BroadcastDistribution takes a BroadcastMode when created.
|
Note
|
BroadcastDistribution is converted to a BroadcastExchangeExec physical operator when EnsureRequirements physical query plan optimization is executed (and enforces partition requirements for data distribution and ordering).
|
createPartitioning Method
|
1 2 3 4 5 |
createPartitioning(numPartitions: Int): Partitioning |
|
Note
|
createPartitioning is part of Distribution Contract to create a Partitioning for a given number of partitions.
|
createPartitioning…FIXME
