spark技术分享GroupState — State Per Group in Stateful Streaming Aggregation
GroupState is the contract for working with a state (of type S) per group for arbitrary stateful aggregation (using mapGroupsWithState or flatMapGroupsWithState operators).
|
Note
|
GroupStateImpl is the one and only implementation of GroupState available.
|
GroupState Contract
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
package org.apache.spark.sql.streaming trait GroupState[S] extends LogicalGroupState[S] { def exists: Boolean def get: S def getOption: Option[S] def update(newState: S): Unit def remove(): Unit def hasTimedOut: Boolean def setTimeoutDuration(durationMs: Long): Unit def setTimeoutDuration(duration: String): Unit def setTimeoutTimestamp(timestampMs: Long): Unit def setTimeoutTimestamp(timestampMs: Long, additionalDuration: String): Unit def setTimeoutTimestamp(timestamp: java.sql.Date): Unit def setTimeoutTimestamp(timestamp: java.sql.Date, additionalDuration: String): Unit } |
| Method | Description |
|---|---|
|
Gives the state |
|
|
Gives the state as |
|
|
Replaces the state with a new state (per group) |
|
