spark技术分享StateStoreHandler Internal Contract
StateStoreHandler is the internal base of state store handlers that manage a StateStore (i.e. commit, abortIfNeeded and metrics).
StateStoreHandler takes a single StateStoreType to be created:
-
KeyToNumValuesTypefor KeyToNumValuesStore -
KeyWithIndexToValueTypefor KeyWithIndexToValueStore
|
Note
|
StateStoreHandler is a Scala private abstract class and cannot be created directly. It is created indirectly for the concrete StateStoreHandlers.
|
| Method | Description | ||
|---|---|---|---|
|
|
|
When requested for the StateStoreMetrics, StateStoreHandler simply requests the StateStore for them.
| StateStoreHandler | Description |
|---|---|
StateStoreHandler takes a single StateStoreType when created.
|
Note
|
StateStoreHandler is a Scala abstract class and cannot be created directly. It is created indirectly when the concrete StateStoreHandlers are.
|
|
Tip
|
Enable Add the following line to
Refer to Logging. |
Committing Changes to State Store — commit Method
|
1 2 3 4 5 |
commit(): Unit |
commit…FIXME
|
Note
|
commit is used when…FIXME
|
abortIfNeeded Method
|
1 2 3 4 5 |
abortIfNeeded(): Unit |
abortIfNeeded…FIXME
|
Note
|
abortIfNeeded is used when…FIXME
|
getStateStore Method
|
1 2 3 4 5 |
getStateStore(keySchema: StructType, valueSchema: StructType): StateStore |
getStateStore…FIXME
|
Note
|
getStateStore is used when KeyToNumValuesStore and KeyWithIndexToValueStore are created.
|
StateStoreType Contract (Sealed Trait)
StateStoreType is required to create a StateStoreHandler.
| StateStoreType | toString | Description |
|---|---|---|
|
|
|
|
|
|
|
|
Note
|
StateStoreType is a Scala private sealed trait which means that all the implementations are in the same compilation unit (a single file).
|
