WebUI — Framework For Web UIs-spark技术分享

WebUI — Base Web UI

WebUI is the base of the web UIs in Apache Spark:

Active Spark applications
Spark History Server
Spark Standalone cluster manager
Spark on Mesos cluster manager

Note	Spark on YARN uses a different web framework for the web UI.

WebUI is used as the parent of WebUITabs.



package org.apache.spark.ui

abstract class WebUI {
  // only required methods that have no implementation
  // the others follow
  def initialize(): Unit
}

package org.apache.spark.ui

abstract class WebUI {

// only required methods that have no implementation

// the others follow

def initialize(): Unit

}

Note	`WebUI` is a `private[spark]` contract.

Method Description

initialize

Used in implementations only to let them initialize their web components

Note	`initialize` does not add anything special to the Scala type hierarchy but a common name to use across `WebUIs` (that could also be possible without it). In other words, `initialize` does not participate in any design pattern or a type hierarchy.

WebUI is a Scala abstract class and cannot be created directly, but only as one of the web UIs.

Table 2. WebUIs
WebUI	Description
HistoryServer	Used in Spark History Server
`MasterWebUI`	Used in Spark Standalone cluster manager
`MesosClusterUI`	Used in Spark on Mesos cluster manager
SparkUI	`WebUI` of a Spark application
`WorkerWebUI`	Used in Spark Standalone cluster manager

Once bound to a Jetty HTTP server, WebUI is available at an HTTP port (and is used in the web URL as boundPort).

WebUI is available at a web URL, i.e. http://[publicHostName]:[boundPort]. The publicHostName is…FIXME and the boundPort is the port that the port the Jetty HTTP Server bound to.

Table 3. WebUI’s Internal Properties (e.g. Registries, Counters and Flags)
Name	Description
`tabs`	WebUITabs Used when…FIXME
`handlers`	`ServletContextHandlers` Used when…FIXME
`pageToHandlers`	`ServletContextHandlers` per WebUIPage Used when…FIXME
`serverInfo`	Optional `ServerInfo` (default: `None`) Used when…FIXME
`publicHostName`	Host name of the UI `publicHostName` is either `SPARK_PUBLIC_DNS` environment variable or `spark.driver.host` configuration property. Defaults to the following if defined (in order): `SPARK_LOCAL_HOSTNAME` environment variable Host name of `SPARK_LOCAL_IP` environment variable `Utils.findLocalInetAddress` Used exclusively when `WebUI` is requested for the web URL
`className`	Used when…FIXME

Tip

Enable INFO or ERROR logging level for the corresponding loggers of the WebUIs, e.g. org.apache.spark.ui.SparkUI, to see what happens inside.

Add the following line to conf/log4j.properties:



log4j.logger.org.apache.spark.ui=INFO

log4j.logger.org.apache.spark.ui=INFO

Refer to Logging.

Creating WebUI Instance

WebUI takes the following when created:

SecurityManager
SSLOptions
Port number
SparkConf
basePath (default: empty)
Name (default: empty)

WebUI initializes the internal registries and counters.

Note	`WebUI` is a Scala abstract class and cannot be created directly, but only as one of the implementations.

Detaching Page And Associated Handlers from UI — `detachPage` Method



detachPage(page: WebUIPage): Unit

detachPage(page: WebUIPage): Unit

detachPage…FIXME

Note	`detachPage` is used when…FIXME

Detaching Tab And Associated Pages from UI — `detachTab` Method



detachTab(tab: WebUITab): Unit

detachTab(tab: WebUITab): Unit

detachTab…FIXME

Note	`detachTab` is used when…FIXME

Detaching Handler — `detachHandler` Method



detachHandler(handler: ServletContextHandler): Unit

detachHandler(handler: ServletContextHandler): Unit

detachHandler…FIXME

Note	`detachHandler` is used when…FIXME

Detaching Handler At Path — `detachHandler` Method



detachHandler(path: String): Unit

detachHandler(path: String): Unit

detachHandler…FIXME

Note	`detachHandler` is used when…FIXME

Attaching Page to UI — `attachPage` Method



attachPage(page: WebUIPage): Unit

attachPage(page: WebUIPage): Unit

Internally, attachPage creates the path of the WebUIPage that is / (forward slash) followed by the prefix of the page.

attachPage creates a HTTP request handler…FIXME

Note	`attachPage` is used when: `WebUI` is requested to attach a WebUITab (the WebUIPages actually) HistoryServer, Spark Standalone’s `MasterWebUI` and `WorkerWebUI`, Spark on Mesos’ `MesosClusterUI` are requested to `initialize`

Attaching Tab And Associated Pages to UI — `attachTab` Method



attachTab(tab: WebUITab): Unit

attachTab(tab: WebUITab): Unit

attachTab attaches every WebUIPage of the input WebUITab.

In the end, attachTab adds the input WebUITab to WebUITab tabs.

Note	`attachTab` is used when…FIXME

Attaching Static Handler — `addStaticHandler` Method



addStaticHandler(resourceBase: String, path: String): Unit

addStaticHandler(resourceBase: String, path: String): Unit

addStaticHandler…FIXME

Note	`addStaticHandler` is used when…FIXME

Attaching Handler to UI — `attachHandler` Method



attachHandler(handler: ServletContextHandler): Unit

attachHandler(handler: ServletContextHandler): Unit

attachHandler simply adds the input Jetty ServletContextHandler to handlers registry and requests the ServerInfo to addHandler (only if the ServerInfo is defined).

Note

attachHandler is used when:

web UIs (i.e. HistoryServer, Spark Standalone’s MasterWebUI and WorkerWebUI, Spark on Mesos’ MesosClusterUI, SparkUI) are requested to initialize
WebUI is requested to attach a page to web UI and addStaticHandler
SparkContext is created (and attaches the driver metrics servlet handler to the web UI)
HistoryServer is requested to attachSparkUI
Spark Standalone’s Master and Worker are requested to onStart (and attach their metrics servlet handlers to the web ui)

`getBasePath` Method



getBasePath: String

getBasePath: String

getBasePath simply returns the base path.

Note	`getBasePath` is used exclusively when `WebUITab` is requested for the base path.

Requesting Header Tabs — `getTabs` Method



getTabs: Seq[WebUITab]

getTabs: Seq[WebUITab]

getTabs simply returns the registered tabs.

Note	`getTabs` is used exclusively when `WebUITab` is requested for the header tabs.

Requesting Handlers — `getHandlers` Method



getHandlers: Seq[ServletContextHandler]

getHandlers: Seq[ServletContextHandler]

getHandlers simply returns the registered handlers.

Note	`getHandlers` is used when…FIXME

Binding UI to Jetty HTTP Server on Host — `bind` Method



bind(): Unit

bind(): Unit

bind…FIXME

Note	`bind` is used when…FIXME

Stopping UI — `stop` Method



stop(): Unit

stop(): Unit

stop…FIXME

Note	`stop` is used when…FIXME

WebUI — Framework For Web UIs

WebUI — Base Web UI

Creating WebUI Instance

Detaching Page And Associated Handlers from UI — `detachPage` Method

Detaching Tab And Associated Pages from UI — `detachTab` Method

Detaching Handler — `detachHandler` Method

Detaching Handler At Path — `detachHandler` Method

Attaching Page to UI — `attachPage` Method

Attaching Tab And Associated Pages to UI — `attachTab` Method

Attaching Static Handler — `addStaticHandler` Method

Attaching Handler to UI — `attachHandler` Method

`getBasePath` Method

Requesting Header Tabs — `getTabs` Method

Requesting Handlers — `getHandlers` Method

Binding UI to Jetty HTTP Server on Host — `bind` Method

Stopping UI — `stop` Method

相关推荐

欢迎关注：spark技术分享

热门标签

近期文章

分类目录

关注公众号：spark技术分享

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

微信扫一扫打赏

QQ咨询

回顶部

WebUI — Base Web UI

Creating WebUI Instance

Detaching Page And Associated Handlers from UI — detachPage Method

Detaching Tab And Associated Pages from UI — detachTab Method

Detaching Handler — detachHandler Method

Detaching Handler At Path — detachHandler Method

Attaching Page to UI — attachPage Method

Attaching Tab And Associated Pages to UI — attachTab Method

Attaching Static Handler — addStaticHandler Method

Attaching Handler to UI — attachHandler Method

getBasePath Method

Requesting Header Tabs — getTabs Method

Requesting Handlers — getHandlers Method

Binding UI to Jetty HTTP Server on Host — bind Method

Stopping UI — stop Method

相关推荐

欢迎关注：spark技术分享

热门标签

近期文章

分类目录

关注公众号：spark技术分享

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

微信扫一扫打赏

QQ咨询

回顶部

Detaching Page And Associated Handlers from UI — `detachPage` Method

Detaching Tab And Associated Pages from UI — `detachTab` Method

Detaching Handler — `detachHandler` Method

Detaching Handler At Path — `detachHandler` Method

Attaching Page to UI — `attachPage` Method

Attaching Tab And Associated Pages to UI — `attachTab` Method

Attaching Static Handler — `addStaticHandler` Method

Attaching Handler to UI — `attachHandler` Method

`getBasePath` Method

Requesting Header Tabs — `getTabs` Method

Requesting Handlers — `getHandlers` Method

Binding UI to Jetty HTTP Server on Host — `bind` Method

Stopping UI — `stop` Method