spark技术分享ContinuousReadSupport Contract — Data Sources with ContinuousReaders
ContinuousReadSupport is the abstraction of data sources with a ContinuousReader.
ContinuousReadSupport defines a single createContinuousReader method that creates a ContinuousReader.
|
1 2 3 4 5 6 7 8 |
ContinuousReader createContinuousReader( Optional<StructType> schema, String checkpointLocation, DataSourceOptions options) |
createContinuousReader is used when:
-
ContinuousExecutionis requested to run a streaming query (and finds ContinuousExecutionRelations in the analyzed logical plan) -
DataStreamReaderis requested to create a streaming query for a ContinuousReadSupport data source
| ContinuousReadSupport | Description |
|---|---|
