spark技术分享ForeachWriter
ForeachWriter is the contract for a foreach writer that is a streaming format that controls streaming writes.
|
Note
|
ForeachWriter is set using foreach operator.
|
|
1 2 3 4 5 6 7 8 9 |
val foreachWriter = new ForeachWriter[String] { ... } streamingQuery. writeStream. foreach(foreachWriter). start |
ForeachWriter Contract
|
1 2 3 4 5 6 7 8 9 10 11 |
package org.apache.spark.sql abstract class ForeachWriter[T] { def open(partitionId: Long, version: Long): Boolean def process(value: T): Unit def close(errorOrNull: Throwable): Unit } |
| Method | Description |
|---|---|
|
Used when… |
|
|
Used when… |
|
|
Used when… |
