ApplicationCache-spark技术分享

ApplicationCache

ApplicationCache is…FIXME

ApplicationCache is created exclusively when HistoryServer is created.

ApplicationCache uses Google Guava 14.0.1 library for the internal appLoader.

Table 1. ApplicationCache’s Internal Properties (e.g. Registries, Counters and Flags)
Name	Description
`appLoader`	Google Guava’s CacheLoader with a custom load which is simply loadApplicationEntry. Used when…FIXME
`removalListener`
`appCache`	Google Guava’s LoadingCache of `CacheKey` keys and `CacheEntry` entries Used when `ApplicationCache` is requested for the following: Cached UI of a Spark application given `appId` and `attemptId` IDs FIXME (other uses)
`metrics`

Creating ApplicationCache Instance

ApplicationCache takes the following when created:

ApplicationCacheOperations
retainedApplications
Clock

ApplicationCache initializes the internal registries and counters.

`loadApplicationEntry` Internal Method


loadApplicationEntry(appId: String, attemptId: Option[String]): CacheEntry

loadApplicationEntry(appId: String, attemptId: Option[String]): CacheEntry

loadApplicationEntry…FIXME

Note	`loadApplicationEntry` is used exclusively when `ApplicationCache` is requested to load a cached entry.

Loading Cached Spark Application UI — `load` Method


load(key: CacheKey): CacheEntry

load(key: CacheKey): CacheEntry

Note	`load` is part of Google Guava’s CacheLoader to retrieve a `CacheEntry`, based on a `CacheKey`, for LoadingCache.

load simply relays to loadApplicationEntry with the appId and attemptId of the input CacheKey.

Requesting Cached UI of Spark Application (CacheEntry) — `get` Method


get(appId: String, attemptId: Option[String] = None): CacheEntry

get(appId: String, attemptId: Option[String] = None): CacheEntry

get…FIXME

Note	`get` is used exclusively when `ApplicationCache` is requested to execute a closure while holding an application’s UI read lock.

Executing Closure While Holding Application’s UI Read Lock — `withSparkUI` Method


withSparkUI[T](appId: String, attemptId: Option[String])(fn: SparkUI => T): T

withSparkUI[T](appId: String, attemptId: Option[String])(fn: SparkUI => T): T

withSparkUI…FIXME

Note	`withSparkUI` is used when `HistoryServer` is requested to withSparkUI and loadAppUi.

ApplicationCache