CodeGenerator-spark技术分享

CodeGenerator

CodeGenerator is a base class for generators of JVM bytecode for expression evaluation.

Table 1. CodeGenerator’s Internal Properties
Name	Description
`cache`	Guava’s LoadingCache with at most 100 pairs of `CodeAndComment` and `GeneratedClass`.
`genericMutableRowType`

Tip

Enable INFO or DEBUG logging level for org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator logger to see what happens inside.

Add the following line to conf/log4j.properties:


log4j.logger.org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator=DEBUG

log4j.logger.org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator=DEBUG

Refer to Logging.

CodeGenerator Contract


package org.apache.spark.sql.catalyst.expressions.codegen

abstract class CodeGenerator[InType, OutType] {
  def create(in: InType): OutType
  def canonicalize(in: InType): InType
  def bind(in: InType, inputSchema: Seq[Attribute]): InType
  def generate(expressions: InType, inputSchema: Seq[Attribute]): OutType
  def generate(expressions: InType): OutType
}

package org.apache.spark.sql.catalyst.expressions.codegen

abstract class CodeGenerator[InType, OutType] {

def create(in: InType): OutType

def canonicalize(in: InType): InType

def bind(in: InType, inputSchema: Seq[Attribute]): InType

def generate(expressions: InType, inputSchema: Seq[Attribute]): OutType

def generate(expressions: InType): OutType

}

Table 2. CodeGenerator Contract
Method	Description
`generate`	Generates an evaluator for expression(s) that may (optionally) have expression(s) bound to a schema (i.e. a collection of Attribute). Used in: `ExpressionEncoder` for UnsafeProjection (for serialization)

Compiling Java Source Code using Janino — `doCompile` Internal Method

Caution

FIXME

Finding or Compiling Java Source Code — `compile` Method

Caution

FIXME

`create` Method


create(references: Seq[Expression]): UnsafeProjection

create(references: Seq[Expression]): UnsafeProjection

Caution

FIXME

Note	`create` is used when: `CodeGenerator` generates an expression evaluator `GenerateOrdering` creates a code gen ordering for `SortOrder` expressions

Creating CodegenContext — `newCodeGenContext` Method


newCodeGenContext(): CodegenContext

newCodeGenContext(): CodegenContext

newCodeGenContext simply creates a new CodegenContext.

Note

newCodeGenContext is used when:

GenerateMutableProjection is requested to create a MutableProjection
GenerateOrdering is requested to create a BaseOrdering
GeneratePredicate is requested to create a Predicate
GenerateSafeProjection is requested to create a Projection
GenerateUnsafeProjection is requested to create a UnsafeProjection
GenerateColumnAccessor is requested to create a ColumnarIterator

CodeGenerator

CodeGenerator

CodeGenerator Contract

Compiling Java Source Code using Janino — `doCompile` Internal Method

Finding or Compiling Java Source Code — `compile` Method

`create` Method

Creating CodegenContext — `newCodeGenContext` Method

相关推荐

欢迎关注：spark技术分享

热门标签

近期文章

分类目录

关注公众号：spark技术分享

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

微信扫一扫打赏

QQ咨询

回顶部

CodeGenerator

CodeGenerator Contract

Compiling Java Source Code using Janino — doCompile Internal Method

Finding or Compiling Java Source Code — compile Method

create Method

Creating CodegenContext — newCodeGenContext Method

相关推荐

欢迎关注：spark技术分享

热门标签

近期文章

分类目录

关注公众号：spark技术分享

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

微信扫一扫打赏

QQ咨询

回顶部

切换注册登录

用户名或邮箱

密码

切换登录注册

昵称

邮箱

Compiling Java Source Code using Janino — `doCompile` Internal Method

Finding or Compiling Java Source Code — `compile` Method

`create` Method

Creating CodegenContext — `newCodeGenContext` Method