spark技术分享ConsoleSinkProvider
ConsoleSinkProvider is a DataSourceV2 with StreamWriteSupport for [spark-sql-streaming-ConsoleSink#< ConsoleSink].
ConsoleSinkProvider is a DataSourceRegister that registers the ConsoleSink streaming sink as console format.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
import org.apache.spark.sql.streaming.Trigger val q = spark .readStream .format("rate") .load .writeStream .format("console") // <-- requests ConsoleSinkProvider for a sink .trigger(Trigger.Once) .start scala> println(q.lastProgress.sink) { "description" : "org.apache.spark.sql.execution.streaming.ConsoleSinkProvider@edde9bb" } |
