spark技术分享 MemoryPlan Logical Operator
MemoryPlan is a leaf logical operator (i.e. LogicalPlan) that is used to query the data that has been written into a MemorySink. MemoryPlan is created when starting continuous writing (to a MemorySink).
|
Tip
|
See the example in MemoryStream. |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
scala> intsOut.explain(true) == Parsed Logical Plan == SubqueryAlias memstream +- MemoryPlan org.apache.spark.sql.execution.streaming.MemorySink@481bf251, [value#21] == Analyzed Logical Plan == value: int SubqueryAlias memstream +- MemoryPlan org.apache.spark.sql.execution.streaming.MemorySink@481bf251, [value#21] == Optimized Logical Plan == MemoryPlan org.apache.spark.sql.execution.streaming.MemorySink@481bf251, [value#21] == Physical Plan == LocalTableScan [value#21] |
When executed, MemoryPlan is translated to LocalTableScanExec physical operator (similar to LocalRelation logical operator) in BasicOperators execution planning strategy.
