spark技术分享OrderedDistribution
OrderedDistribution is a Distribution that…FIXME
OrderedDistribution specifies None for the required number of partitions.
|
Note
|
None for the required number of partitions indicates to use any number of partitions (possibly spark.sql.shuffle.partitions configuration property with the default of 200 partitions).
|
OrderedDistribution is created when…FIXME
OrderedDistribution takes SortOrder expressions for ordering when created.
OrderedDistribution requires that the ordering expressions should not be empty (i.e. Nil).
createPartitioning Method
|
1 2 3 4 5 |
createPartitioning(numPartitions: Int): Partitioning |
|
Note
|
createPartitioning is part of Distribution Contract to create a Partitioning for a given number of partitions.
|
createPartitioning…FIXME
