当前位置：spark技术分享 > structured-streaming > 正文

StreamingRelationV2 Leaf Logical Operator

2015-09-14 分类：structured-streaming 阅读(3280) 评论(0)

StreamingRelationV2 Leaf Logical Operator

StreamingRelationV2 is a MultiInstanceRelation leaf logical operator that is used to represent MicroBatchReadSupport or ContinuousReadSupport streaming data sources in a logical plan of a streaming query.

Tip	Find out more on leaf logical operators in LeafNode — Base Logical Operator with No Child Operators and Optional Statistics section in The Internals of Spark SQL book.

StreamingRelationV2 is created when:

ContinuousMemoryStream is created
DataStreamReader is requested to create a streaming query for a MicroBatchReadSupport or a ContinuousReadSupport data source

StreamingRelationV2 is resolved (replaced) to the following leaf logical operators:

ContinuousExecutionRelation when ContinuousExecution is requested for the analyzed logical plan
StreamingExecutionRelation when MicroBatchExecution is requested for the analyzed logical plan

Creating StreamingRelationV2 Instance

StreamingRelationV2 takes the following to be created:

DataSourceV2
Name of the data source
Options (Map[String, String])
Output attributes (Seq[Attribute])
Optional StreamingRelation
SparkSession

赞(0) 打赏

未经允许不得转载：spark技术分享 » StreamingRelationV2 Leaf Logical Operator

标签：structured-streaming

相关推荐

关注公众号：spark技术分享

联系我们联系我们

QQ咨询
QQ咨询
回顶
回顶部