spark技术分享AllJobsPage — Showing All Jobs in Web UI
AllJobsPage renders a summary, an event timeline, and active, completed, and failed jobs of a Spark application.
|
Tip
|
Jobs (in any state) are displayed when their number is greater than 0.
|
AllJobsPage displays the Summary section with the current Spark user, total uptime, scheduling mode, and the number of jobs per status.
|
Note
|
AllJobsPage uses JobProgressListener for Scheduling Mode.
|
Figure 1. Summary Section in Jobs Tab
Under the summary section is the Event Timeline section.
Figure 2. Event Timeline in Jobs Tab
|
Note
|
AllJobsPage uses ExecutorsListener to build the event timeline. |
Active Jobs, Completed Jobs, and Failed Jobs sections follow.
Figure 3. Job Status Section in Jobs Tab
Jobs are clickable, i.e. you can click on a job to see information about the stages of tasks inside it.
When you hover over a job in Event Timeline not only you see the job legend but also the job is highlighted in the Summary section.
Figure 4. Hovering Over Job in Event Timeline Highlights The Job in Status Section
The Event Timeline section shows not only jobs but also executors.
Figure 5. Executors in Event Timeline
|
Tip
|
Use Programmable Dynamic Allocation (using SparkContext) to manage executors for demo purposes.
|
