spark技术分享RuntimeReplaceable Contract — Replaceable SQL Expressions
RuntimeReplaceable is the marker contract for unary expressions that are replaced by Catalyst Optimizer with their child expression (that can then be evaluated).
|
Note
|
Catalyst Optimizer uses ReplaceExpressions logical optimization to replace RuntimeReplaceable expressions.
|
RuntimeReplaceable contract allows for expression aliases, i.e. expressions that are fairly complex in the inside than on the outside, and is used to provide compatibility with other SQL databases by supporting SQL functions with their more complex Catalyst expressions (that are already supported by Spark SQL).
|
Note
|
RuntimeReplaceables are tied up to their SQL functions in FunctionRegistry. |
RuntimeReplaceable expressions cannot be evaluated (i.e. produce a value given an internal row) and therefore have to be replaced in the query execution pipeline.
|
1 2 3 4 5 6 7 8 9 10 |
package org.apache.spark.sql.catalyst.expressions trait RuntimeReplaceable extends UnaryExpression with Unevaluable { // as a marker contract it only marks a class // no methods are required } |
|
Note
|
To make sure the explain plan and expression SQL works correctly, a RuntimeReplaceable implementation should override flatArguments and sql methods.
|
| RuntimeReplaceable | Standard Function | SQL Function |
|---|---|---|
