spark技术分享StreamingRelationV2 Leaf Logical Operator
StreamingRelationV2 is a MultiInstanceRelation leaf logical operator that is used to represent MicroBatchReadSupport or ContinuousReadSupport streaming data sources in a logical plan of a streaming query.
|
Tip
|
Find out more on leaf logical operators in LeafNode — Base Logical Operator with No Child Operators and Optional Statistics section in The Internals of Spark SQL book. |
StreamingRelationV2 is created when:
-
ContinuousMemoryStream is created
-
DataStreamReaderis requested to create a streaming query for a MicroBatchReadSupport or a ContinuousReadSupport data source
StreamingRelationV2 is resolved (replaced) to the following leaf logical operators:
-
ContinuousExecutionRelation when
ContinuousExecutionis requested for the analyzed logical plan -
StreamingExecutionRelation when
MicroBatchExecutionis requested for the analyzed logical plan
Creating StreamingRelationV2 Instance
StreamingRelationV2 takes the following to be created:
-
Optional StreamingRelation
