WindowFunctionFrame-spark技术分享

WindowFunctionFrame

WindowFunctionFrame is a contract for…FIXME

Table 1. WindowFunctionFrame’s Implementations
Name	Description
`OffsetWindowFunctionFrame`
`SlidingWindowFunctionFrame`
`UnboundedFollowingWindowFunctionFrame`
`UnboundedPrecedingWindowFunctionFrame`
UnboundedWindowFunctionFrame

`UnboundedWindowFunctionFrame`

UnboundedWindowFunctionFrame is a WindowFunctionFrame that gives the same value for every row in a partition.

UnboundedWindowFunctionFrame is created for AggregateFunctions (in AggregateExpressions) or AggregateWindowFunctions with no frame defined (i.e. no rowsBetween or rangeBetween) that boils down to using the entire partition frame.

UnboundedWindowFunctionFrame takes the following when created:

Target InternalRow
AggregateProcessor

`prepare` Method



prepare(rows: ExternalAppendOnlyUnsafeRowArray): Unit

prepare(rows: ExternalAppendOnlyUnsafeRowArray): Unit

prepare requests AggregateProcessor to initialize passing in the number of UnsafeRows in the input ExternalAppendOnlyUnsafeRowArray.

prepare then requests ExternalAppendOnlyUnsafeRowArray to generate an interator.

In the end, prepare requests AggregateProcessor to update passing in every UnsafeRow in the iterator one at a time.

`write` Method



write(index: Int, current: InternalRow): Unit

write(index: Int, current: InternalRow): Unit

write simply requests AggregateProcessor to evaluate the target InternalRow.

WindowFunctionFrame Contract



package org.apache.spark.sql.execution.window

abstract class WindowFunctionFrame {
  def prepare(rows: ExternalAppendOnlyUnsafeRowArray): Unit
  def write(index: Int, current: InternalRow): Unit
}

package org.apache.spark.sql.execution.window

abstract class WindowFunctionFrame {

def prepare(rows: ExternalAppendOnlyUnsafeRowArray): Unit

def write(index: Int, current: InternalRow): Unit

}

Note	`WindowFunctionFrame` is a `private[window]` contract.

Table 2. WindowFunctionFrame Contract
Method	Description
`prepare`	Used exclusively when `WindowExec` operator fetches all UnsafeRows for a partition (passing in ExternalAppendOnlyUnsafeRowArray with all `UnsafeRows`).
`write`	Used exclusively when the Iterator[InternalRow] (from executing `WindowExec`) is requested a next row.

WindowFunctionFrame