TaskInfo-spark技术分享

TaskInfo

TaskInfo is information about a running task attempt inside a TaskSet.

TaskInfo is created when:

Note	Back then, at the commit 63051dd2bcc4bf09d413ff7cf89a37967edc33ba, when `TaskInfo` was first merged to Apache Spark on 07/06/12, `TaskInfo` was part of `spark.scheduler.mesos` package — note “Mesos” in the name of the package that shows how much Spark and Mesos influenced each other at that time.

Table 1. TaskInfo’s Internal Registries and Counters
Name	Description
`finishTime`	Time when `TaskInfo` was marked as finished. Used when…FIXME

TaskInfo takes the following when created:

Task ID
Index of the task within its TaskSet that may not necessarily be the same as the ID of the RDD partition that the task is computing.
Task attempt ID
Time when the task was dequeued for execution
Executor that has been offered (as a resource) to run the task
Host of the executor
TaskLocality, i.e. locality preference of the task
Flag whether a task is speculative or not



markFinished(state: TaskState, time: Long = System.currentTimeMillis): Unit

markFinished(state: TaskState, time: Long = System.currentTimeMillis): Unit

markFinished records the input time as finishTime.

markFinished marks TaskInfo as failed when the input state is FAILED or killed for state being KILLED.

Note	`markFinished` is used when `TaskSetManager` is notified that a task has finished successfully or failed.