关注 spark技术分享,
撸spark源码 玩spark最佳实践

StreamingQuery

StreamingQuery

StreamingQuery is the contract for a streaming query that is executed continuously and concurrently (i.e. on a separate thread).

Note
StreamingQuery is called continuous query or streaming query.
Note
StreamingQuery is a Scala trait with the only implementation being StreamExecution (and less importanly StreamingQueryWrapper for serializing a non-serializable StreamExecution).

StreamingQuery can be in two states:

  • active (started)

  • inactive (stopped)

If inactive, StreamingQuery may have transitioned into the state due to an StreamingQueryException (that is available under exception).

StreamingQuery tracks current state of all the sources, i.e. SourceStatus, as sourceStatuses.

There could only be a single Sink for a StreamingQuery with many Sources.

StreamingQuery can be stopped by stop or an exception.

Table 1. StreamingQuery Contract
Method Description

awaitTermination

Used when…​FIXME

exception

StreamingQueryException if the query has finished due to an exception

Used when…​FIXME

explain

Used when…​FIXME

id

The unique identifier of the streaming query

Used when…​FIXME

isActive

Used when…​FIXME

lastProgress

The last StreamingQueryProgress of the streaming query

Used when…​FIXME

name

The name of the query that is unique across all active queries

Used when…​FIXME

processAllAvailable

Waits the streaming query until there are no data available in sources or the query has been terminated.

Used when…​FIXME

recentProgress

Collection of the recent StreamingQueryProgress updates.

Used when…​FIXME

runId

The unique identifier of the current execution of the streaming query

Used when…​FIXME

sparkSession

Used when…​FIXME

status

The StreamingQueryStatus of the streaming query (as StreamExecution has accumulated being a ProgressReporter while running the streaming query)

Used when…​FIXME

stop

Stops the streaming query

Used exclusively when StreamingQueryWrapper is requested to stop

赞(0) 打赏
未经允许不得转载:spark技术分享 » StreamingQuery
分享到: 更多 (0)

关注公众号:spark技术分享

联系我们联系我们

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

微信扫一扫打赏