spark技术分享MetricsSystem — Registry of Metrics Sources and Sinks of Spark Subsystem
MetricsSystem is a registry of metrics sources and sinks of a Spark subsystem, e.g. the driver of a Spark application.
MetricsSystem may have at most one MetricsServlet JSON metrics sink (which is registered by default).
When created, MetricsSystem requests MetricsConfig to initialize.
| Name | When Created |
|---|---|
|
|
Spark Standalone’s |
|
|
|
|
|
|
|
|
Spark Standalone’s |
|
|
Spark on Mesos’ |
|
|
|
|
|
Spark Standalone’s |
MetricsSystem uses MetricRegistry as the integration point to Dropwizard Metrics library.
| Name | Description |
|---|---|
|
Initialized when |
|
|
MetricsServlet JSON metrics sink that is only available for the metrics instances with a web UI, i.e. the driver of a Spark application and Spark Standalone’s Initialized when Used exclusively when |
|
|
Dropwizard Metrics’ MetricRegistry Used when |
|
|
Flag that indicates whether Default: |
|
|
Metrics sinks in a Spark application. Used when |
|
|
Metrics sources in a Spark application. Used when |
|
Tip
|
Enable Add the following line to
Refer to Logging. |
“Static” Metrics Sources for Spark SQL — StaticSources
|
Caution
|
FIXME |
Registering Metrics Source — registerSource Method
|
1 2 3 4 5 |
registerSource(source: Source): Unit |
registerSource adds source to sources internal registry.
registerSource creates an identifier for the metrics source and registers it with MetricRegistry.
|
Note
|
registerSource uses Metrics’ MetricRegistry.register to register a metrics source under a given name.
|
When registerSource tries to register a name more than once, you should see the following INFO message in the logs:
|
1 2 3 4 5 |
INFO Metrics already registered |
|
Note
|
|
Building Metrics Source Identifier — buildRegistryName Method
|
1 2 3 4 5 |
buildRegistryName(source: Source): String |
|
Note
|
buildRegistryName is used to build the metrics source identifiers for a Spark application’s driver and executors, but also for other Spark framework’s components (e.g. Spark Standalone’s master and workers).
|
|
Note
|
buildRegistryName uses spark.metrics.namespace and spark.executor.id Spark properties to differentiate between a Spark application’s driver and executors, and the other Spark framework’s components.
|
(only when instance is driver or executor) buildRegistryName builds metrics source name that is made up of spark.metrics.namespace, spark.executor.id and the name of the source.
|
Note
|
buildRegistryName uses Dropwizard Metrics’ MetricRegistry to build metrics source identifiers.
|
|
Caution
|
FIXME Finish for the other components. |
Registering Metrics Sources for Spark Instance — registerSources Internal Method
|
1 2 3 4 5 |
registerSources(): Unit |
registerSources finds metricsConfig configuration for the metrics instance.
|
Note
|
instance is defined when MetricsSystem is created.
|
registerSources finds the configuration of all the metrics sources for the subsystem (as described with source. prefix).
For every metrics source, registerSources finds class property, creates an instance, and in the end registers it.
When registerSources fails, you should see the following ERROR message in the logs followed by the exception.
|
1 2 3 4 5 |
ERROR Source class [classPath] cannot be instantiated |
|
Note
|
registerSources is used exclusively when MetricsSystem is started.
|
Requesting JSON Servlet Handler — getServletHandlers Method
|
1 2 3 4 5 |
getServletHandlers: Array[ServletContextHandler] |
If the MetricsSystem is running and the MetricsServlet is defined for the metrics system, getServletHandlers simply requests the MetricsServlet for the JSON servlet handler.
When MetricsSystem is not running getServletHandlers throws an IllegalArgumentException.
|
1 2 3 4 5 |
Can only call getServletHandlers on a running MetricsSystem |
|
Note
|
|
Registering Metrics Sinks — registerSinks Internal Method
|
1 2 3 4 5 |
registerSinks(): Unit |
registerSinks requests the MetricsConfig for the configuration of the instance.
registerSinks requests the MetricsConfig for the configuration of all metrics sinks (i.e. configuration entries that match ^sink\\.(.)\\.(.) regular expression).
For every metrics sink configuration, registerSinks takes class property and (if defined) creates an instance of the metric sink using an constructor that takes the configuration, MetricRegistry and SecurityManager.
For a single servlet metrics sink, registerSinks converts the sink to a MetricsServlet and sets the metricsServlet internal registry.
For all other metrics sinks, registerSinks adds the sink to the sinks internal registry.
In case of an Exception, registerSinks prints out the following ERROR message to the logs:
|
1 2 3 4 5 |
Sink class [classPath] cannot be instantiated |
|
Note
|
registerSinks is used exclusively when MetricsSystem is requested to start.
|
getSourcesByName Method
|
1 2 3 4 5 |
getSourcesByName(sourceName: String): Seq[Source] |
getSourcesByName…FIXME
|
Note
|
getSourcesByName is used when…FIXME
|
removeSource Method
|
1 2 3 4 5 |
removeSource(source: Source): Unit |
removeSource…FIXME
|
Note
|
removeSource is used when…FIXME
|
Creating MetricsSystem Instance
MetricsSystem takes the following when created:
MetricsSystem initializes the internal registries and counters.
When created, MetricsSystem requests MetricsConfig to initialize.
|
Note
|
createMetricsSystem is used to create a new MetricsSystems instance instead.
|
Creating MetricsSystem Instance For Subsystem — createMetricsSystem Factory Method
|
1 2 3 4 5 6 7 8 |
createMetricsSystem( instance: String conf: SparkConf securityMgr: SecurityManager): MetricsSystem |
createMetricsSystem returns a new MetricsSystem.
|
Note
|
createMetricsSystem is used when a metrics instance is created.
|
Requesting Sinks to Report Metrics — report Method
|
1 2 3 4 5 |
report(): Unit |
report simply requests the registered metrics sinks to report metrics.
|
Note
|
report is used when SparkContext, Executor, Spark Standalone’s Master and Worker, Spark on Mesos’ MesosClusterScheduler are requested to stop
|
Starting MetricsSystem — start Method
|
1 2 3 4 5 |
start(): Unit |
start turns running flag on.
|
Note
|
start can only be called once and throws an IllegalArgumentException when called multiple times.
|
start registers the “static” metrics sources for Spark SQL, i.e. CodegenMetrics and HiveCatalogMetrics.
start then registers the configured metrics sources and sinks for the Spark instance.
In the end, start requests the registered metrics sinks to start.
start throws an IllegalArgumentException when running flag is on.
|
1 2 3 4 5 |
requirement failed: Attempting to start a MetricsSystem that is already running |
