spark技术分享StreamWriteSupport Contract — DataSourceV2 Sinks with Custom StreamWriter
StreamWriteSupport is the abstraction of DataSourceV2 sinks that can create a StreamWriter (when used in streaming queries in MicroBatchExecution and ContinuousExecution).
|
1 2 3 4 5 6 7 8 9 |
StreamWriter createStreamWriter( String queryId, StructType schema, OutputMode mode, DataSourceOptions options) |
createStreamWriter is used when the stream execution thread for a streaming query is started and requests the stream execution engines to start, i.e.
-
ContinuousExecutionis requested to runContinuous -
MicroBatchExecutionis requested to run a single streaming batch
| StreamWriteSupport | Description |
|---|---|
