spark技术分享SQL Tab — Monitoring Structured Queries in web UI
SQL tab in web UI shows SQLMetrics per physical operator in a structured query physical plan.
You can access the SQL tab under /SQL URL, e.g. http://localhost:4040/SQL/.
By default, it displays all SQL query executions. However, after a query has been selected, the SQL tab displays the details for the structured query execution.
AllExecutionsPage
AllExecutionsPage displays all SQL query executions in a Spark application per state sorted by their submission time reversed.
Internally, the page requests SQLListener for query executions in running, completed, and failed states (the states correspond to the respective tables on the page).
ExecutionPage — Details for Query
ExecutionPage shows details for structured query execution by id.
|
Note
|
The id request parameter is mandatory.
|
ExecutionPage displays a summary with Submitted Time, Duration, the clickable identifiers of the Running Jobs, Succeeded Jobs, and Failed Jobs.
It also display a visualization (using accumulator updates and the SparkPlanGraph for the query) with the expandable Details section (that corresponds to SQLExecutionUIData.physicalPlanDescription).
If there is no information to display for a given query id, you should see the following page.
Internally, it uses SQLListener exclusively to get the SQL query execution metrics. It requests SQLListener for SQL execution data to display for the id request parameter.
Creating SQLTab Instance
SQLTab is created when SharedState is or at the first SparkListenerSQLExecutionStart event when Spark History Server is used.
|
Note
|
SharedState represents the shared state across SparkSessions.
|
