spark技术分享TaskLocation
TaskLocation is a location where a task should run.
TaskLocation can either be a host alone or a (host, executorID) pair (as ExecutorCacheTaskLocation).
With ExecutorCacheTaskLocation the Spark scheduler prefers to launch the task on the given executor, but the next level of preference is any executor on the same host if this is not possible.
|
Note
|
TaskLocation is a Scala private[spark] sealed trait (i.e. all the available implementations of TaskLocation trait are in a single Scala file).
|
| Name | Description |
|---|---|
|
A location on a host. |
|
|
A location that includes both a host and an executor id on that host. |
|
|
A location on a host that is cached by Hadoop HDFS. Used exclusively when HadoopRDD and NewHadoopRDD are requested for their placement preferences (aka preferred locations). |
