ExecutorBackend — Pluggable Executor Backends-spark技术分享

ExecutorBackend — Pluggable Executor Backends

Figure 1. ExecutorBackend receives notifications from TaskRunners

Note	`TaskRunner` manages a single individual task and is managed by an `Executor` to launch a task.

Caution

FIXME What is “a scheduler” in this context?

It is effectively a bridge between the driver and an executor, i.e. there are two endpoints running.

There are three concrete executor backends:



trait ExecutorBackend {
  def statusUpdate(taskId: Long, state: TaskState, data: ByteBuffer): Unit
}

trait ExecutorBackend {

def statusUpdate(taskId: Long, state: TaskState, data: ByteBuffer): Unit

}

Note	`ExecutorBackend` is a `private[spark]` contract.

Table 1. ExecutorBackend Contract
Method	Description
`statusUpdate`	Used when `TaskRunner` is requested to run a task (to send task status updates).