spark技术分享StreamingRelationExec Leaf Physical Operator
StreamingRelationExec is a leaf physical operator (i.e. LeafExecNode) that…FIXME
StreamingRelationExec is created when StreamingRelationStrategy plans StreamingRelation and StreamingExecutionRelation logical operators.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
scala> spark.version res0: String = 2.3.0-SNAPSHOT val rates = spark. readStream. format("rate"). load // StreamingRelation logical operator scala> println(rates.queryExecution.logical.numberedTreeString) 00 StreamingRelation DataSource(org.apache.spark.sql.SparkSession@31ba0af0,rate,List(),None,List(),None,Map(),None), rate, [timestamp#0, value#1L] // StreamingRelationExec physical operator (shown without "Exec" suffix) scala> rates.explain == Physical Plan == StreamingRelation rate, [timestamp#0, value#1L] |
StreamingRelationExec is not supposed to be executed and is used…FIXME
Creating StreamingRelationExec Instance
StreamingRelationExec takes the following when created:
-
The name of a streaming data source
