spark技术分享StreamingQueryListenerBus — Notification Bus for Streaming Events
StreamingQueryListenerBus is an event bus (i.e. ListenerBus) to dispatch streaming events to StreamingQueryListener streaming event listeners.
StreamingQueryListenerBus is created when StreamingQueryManager is created (as the internal listenerBus).
StreamingQueryListenerBus is also a SparkListener and registers itself with LiveListenerBus to intercept a QueryStartedEvent.
| Name | Description |
|---|---|
|
Collection of active streaming queries by their runIds.
Used mainly when |
Posting StreamingQueryListener Events to LiveListenerBus — post Method
|
1 2 3 4 5 |
post(event: StreamingQueryListener.Event): Unit |
post simply posts the input event straight to LiveListenerBus except QueryStartedEvent events.
For QueryStartedEvent events, post adds the query’s runId to activeQueryRunIds registry first before posting the event to LiveListenerBus followed by postToAll.
|
Note
|
post is used exclusively when StreamingQueryManager posts StreamingQueryListener event.
|
Creating StreamingQueryListenerBus Instance
StreamingQueryListenerBus takes the following when created:
StreamingQueryListenerBus registers itself with LiveListenerBus.
StreamingQueryListenerBus initializes the internal registries and counters.
