TriggerExecutor-spark技术分享

TriggerExecutor

TriggerExecutor is the interface for trigger executors that StreamExecution uses to execute a batch runner.

Note	Batch runner is an executable code that is executed at regular intervals. It is also called a trigger handler.



package org.apache.spark.sql.execution.streaming

trait TriggerExecutor {
  def execute(batchRunner: () => Boolean): Unit
}

package org.apache.spark.sql.execution.streaming

trait TriggerExecutor {

def execute(batchRunner: () => Boolean): Unit

}

Note	`StreamExecution` reports a `IllegalStateException` when TriggerExecutor is different from the two built-in implementations: `OneTimeExecutor` or `ProcessingTimeExecutor`.

TriggerExecutor Description

OneTimeExecutor

Executes batchRunner exactly once.

ProcessingTimeExecutor

Executes batchRunner at regular intervals (as defined using ProcessingTime and DataStreamWriter.trigger method).



ProcessingTimeExecutor(processingTime: ProcessingTime, clock: Clock = new SystemClock())

ProcessingTimeExecutor(processingTime: ProcessingTime, clock: Clock = new SystemClock())

Note	Processing terminates when `batchRunner` returns `false`.

Caution

FIXME