spark技术分享ExecutionListenerManager — Management Interface of QueryExecutionListeners
ExecutionListenerManager is the management interface for QueryExecutionListeners that listen for execution metrics:
-
Name of the action (that triggered a query execution)
-
Execution time of this query (in nanoseconds)
ExecutionListenerManager is available as listenerManager property of SparkSession (and listenerManager property of SessionState).
|
1 2 3 4 5 6 7 8 9 |
scala> :type spark.listenerManager org.apache.spark.sql.util.ExecutionListenerManager scala> :type spark.sessionState.listenerManager org.apache.spark.sql.util.ExecutionListenerManager |
ExecutionListenerManager takes a single SparkConf when created
While created, ExecutionListenerManager reads spark.sql.queryExecutionListeners configuration property with QueryExecutionListeners and registers them.
ExecutionListenerManager uses spark.sql.queryExecutionListeners configuration property as the list of QueryExecutionListeners that should be automatically added to newly created sessions (and registers them while being created).
| Method | Description | ||
|---|---|---|---|
|
|||
|
|||
|
ExecutionListenerManager is created exclusively when BaseSessionStateBuilder is requested for ExecutionListenerManager (while SessionState is built).
ExecutionListenerManager uses listeners internal registry for registered QueryExecutionListeners.
onSuccess Internal Method
|
1 2 3 4 5 |
onSuccess(funcName: String, qe: QueryExecution, duration: Long): Unit |
onSuccess…FIXME
|
Note
|
|
onFailure Internal Method
|
1 2 3 4 5 |
onFailure(funcName: String, qe: QueryExecution, exception: Exception): Unit |
onFailure…FIXME
|
Note
|
|
withErrorHandling Internal Method
|
1 2 3 4 5 |
withErrorHandling(f: QueryExecutionListener => Unit): Unit |
withErrorHandling…FIXME
Registering QueryExecutionListener — register Method
|
1 2 3 4 5 |
register(listener: QueryExecutionListener): Unit |
Internally, register simply registers (adds) the input QueryExecutionListener to the listeners internal registry.
