spark技术分享DataWritingCommandExec Physical Operator
DataWritingCommandExec is a physical operator that is the execution environment for a DataWritingCommand logical command at execution time.
DataWritingCommandExec is created exclusively when BasicOperators execution planning strategy is requested to plan a DataWritingCommand logical command.
When requested for performance metrics, DataWritingCommandExec simply requests the DataWritingCommand for them.
| Name | Description |
|---|---|
|
|
Collection of InternalRows ( Used when |
Executing Physical Operator and Collecting Results — executeCollect Method
|
1 2 3 4 5 |
executeCollect(): Array[InternalRow] |
|
Note
|
executeCollect is part of the SparkPlan Contract to execute the physical operator and collect results.
|
executeCollect…FIXME
executeToIterator Method
|
1 2 3 4 5 |
executeToIterator: Iterator[InternalRow] |
|
Note
|
executeToIterator is part of the SparkPlan Contract to…FIXME.
|
executeToIterator…FIXME
Taking First N UnsafeRows — executeTake Method
|
1 2 3 4 5 |
executeTake(limit: Int): Array[InternalRow] |
|
Note
|
executeTake is part of the SparkPlan Contract to take the first n UnsafeRows.
|
executeTake…FIXME
Executing Physical Operator (Generating RDD[InternalRow]) — doExecute Method
|
1 2 3 4 5 |
doExecute(): RDD[InternalRow] |
|
Note
|
doExecute is part of the SparkPlan Contract to generate the runtime representation of a structured query as a distributed computation over internal binary rows on Apache Spark (i.e. RDD[InternalRow]).
|
doExecute simply requests the SQLContext for the SparkContext that is then requested to distribute (parallelize) the sideEffectResult (over 1 partition).
