spark技术分享Web UI — Spark Application’s Web Console
Web UI (aka Application UI or webUI or Spark UI) is the web interface of a Spark application to monitor and inspect Spark job executions in a web browser.
Figure 1. Welcome page – Jobs page
Every time you create a SparkContext in a Spark application you also launch an instance of web UI. The web UI is available at http://[driverHostname]:4040 by default.
|
Note
|
The default port can be changed using spark.ui.port configuration property. SparkContext will increase the port if it is already taken until an open port is found.
|
web UI comes with the following tabs (which may not all be visible immediately, but only after the respective modules are in use, e.g. the SQL or Streaming tabs):
|
Tip
|
You can use the web UI after the application has finished by persisting events (using EventLoggingListener) and using Spark History Server. |
|
Note
|
All the information that is displayed in web UI is available thanks to JobProgressListener and other SparkListeners. One could say that web UI is a web layer to Spark listeners. |
