spark技术分享UDFRegistration — Session-Scoped FunctionRegistry
UDFRegistration is an interface to the session-scoped FunctionRegistry to register user-defined functions (UDFs) and user-defined aggregate functions (UDAFs).
UDFRegistration is available using SparkSession.
|
1 2 3 4 5 6 7 |
import org.apache.spark.sql.SparkSession val spark: SparkSession = ... spark.udf |
UDFRegistration takes a FunctionRegistry when created.
UDFRegistration is created exclusively for SessionState.
Registering UserDefinedFunction (with FunctionRegistry) — register Method
|
1 2 3 4 5 6 7 8 |
register(name: String, func: Function0[RT]): UserDefinedFunction register(name: String, func: Function1[A1, RT]): UserDefinedFunction ... register(name: String, func: Function22[A1, A2, A3, A4, A5, A6, A7, A8, A9, A10, A11, A12, A13, A14, A15, A16, A17, A18, A19, A20, A21, A22, RT]): UserDefinedFunction |
register…FIXME
|
Note
|
register is used when…FIXME
|
Registering UserDefinedFunction (with FunctionRegistry) — register Method
|
1 2 3 4 5 |
register(name: String, udf: UserDefinedFunction): UserDefinedFunction |
register…FIXME
|
Note
|
register is used when…FIXME
|
Registering UserDefinedAggregateFunction (with FunctionRegistry) — register Method
|
1 2 3 4 5 6 7 |
register( name: String, udaf: UserDefinedAggregateFunction): UserDefinedAggregateFunction |
register registers a UserDefinedAggregateFunction under name with FunctionRegistry.
register creates a ScalaUDAF internally to register a UDAF.
|
Note
|
register gives the input udaf aggregate function back after the function has been registered with FunctionRegistry.
|
