spark技术分享TypedColumn
TypedColumn is a Column with the ExpressionEncoder for the types of the input and the output.
|
1 2 3 4 5 6 7 8 9 |
scala> val id = $"id".as[Int] id: org.apache.spark.sql.TypedColumn[Any,Int] = id scala> id.expr res1: org.apache.spark.sql.catalyst.expressions.Expression = 'id |
name Operator
|
1 2 3 4 5 |
name(alias: String): TypedColumn[T, U] |
|
Note
|
name is part of Column Contract to…FIXME.
|
name…FIXME
|
Note
|
name is used when…FIXME
|
Creating TypedColumn — withInputType Internal Method
|
1 2 3 4 5 6 7 |
withInputType( inputEncoder: ExpressionEncoder[_], inputAttributes: Seq[Attribute]): TypedColumn[T, U] |
withInputType…FIXME
|
Note
|
|
Creating TypedColumn Instance
TypedColumn takes the following when created:
-
Catalyst expression
-
ExpressionEncoder of the column results
TypedColumn initializes the internal registries and counters.
