TypedImperativeAggregate-spark技术分享

TypedImperativeAggregate — Contract for Imperative Aggregate Functions with Custom Aggregation Buffer

TypedImperativeAggregate is the contract for imperative aggregation functions that allows for an arbitrary user-defined java object to be used as internal aggregation buffer.

Table 1. TypedImperativeAggregate as ImperativeAggregate
ImperativeAggregate Method	Description
`aggBufferAttributes`
`aggBufferSchema`
initialize	Creates an aggregation buffer and puts it at mutableAggBufferOffset position in the input `buffer` InternalRow.
`inputAggBufferAttributes`

Table 2. TypedImperativeAggregate’s Direct Implementations
Name	Description
`ApproximatePercentile`
`Collect`
ComplexTypedAggregateExpression
`CountMinSketchAgg`
`HiveUDAFFunction`
`Percentile`

TypedImperativeAggregate Contract



package org.apache.spark.sql.catalyst.expressions.aggregate

abstract class TypedImperativeAggregate[T] extends ImperativeAggregate {
  def createAggregationBuffer(): T
  def deserialize(storageFormat: Array[Byte]): T
  def eval(buffer: T): Any
  def merge(buffer: T, input: T): T
  def serialize(buffer: T): Array[Byte]
  def update(buffer: T, input: InternalRow): T
}

package org.apache.spark.sql.catalyst.expressions.aggregate

abstract class TypedImperativeAggregate[T] extends ImperativeAggregate {

def createAggregationBuffer(): T

def deserialize(storageFormat: Array[Byte]): T

def eval(buffer: T): Any

def merge(buffer: T, input: T): T

def serialize(buffer: T): Array[Byte]

def update(buffer: T, input: InternalRow): T

}

Table 3. TypedImperativeAggregate Contract
Method	Description
`createAggregationBuffer`	Used exclusively when a `TypedImperativeAggregate` is initialized
`deserialize`
`eval`
`merge`
`serialize`
`update`

TypedImperativeAggregate

TypedImperativeAggregate — Contract for Imperative Aggregate Functions with Custom Aggregation Buffer

TypedImperativeAggregate Contract

相关推荐

欢迎关注：spark技术分享

热门标签

近期文章

分类目录

关注公众号：spark技术分享

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

微信扫一扫打赏

QQ咨询

回顶部