spark技术分享MapPartitionsRDD
MapPartitionsRDD is an RDD that applies the provided function f to every partition of the parent RDD.
By default, it does not preserve partitioning — the last input parameter preservesPartitioning is false. If it is true, it retains the original RDD’s partitioning.
MapPartitionsRDD is the result of the following transformations:
-
map -
flatMap -
filter -
glom -
mapPartitionsWithIndex
