JobListener

2014-08-17 分类：spark-core 阅读(1318) 评论(0)

Spark subscribes for job completion or failure events (after submitting a job to DAGScheduler) using JobListener trait.

The following are the job listeners used:

JobWaiter waits until DAGScheduler completes a job and passes the results of tasks to a resultHandler function.
ApproximateActionListener…FIXME

An instance of JobListener is used in the following places:

In ActiveJob as a listener to notify if tasks in this job finish or the job fails.
In JobSubmitted

JobListener is a private[spark] contract with the following two methods:



private[spark] trait JobListener {
  def taskSucceeded(index: Int, result: Any)
  def jobFailed(exception: Exception)
}

private[spark] trait JobListener {

def taskSucceeded(index: Int, result: Any)

def jobFailed(exception: Exception)

}

A JobListener object is notified each time a task succeeds (by taskSucceeded) and when the whole job fails (by jobFailed).

赞(0) 打赏

未经允许不得转载：spark技术分享 » JobListener

关注公众号：spark技术分享