spark技术分享UnixTimestamp TimeZoneAware Binary Expression
UnixTimestamp is a binary expression with timezone support that represents unix_timestamp function (and indirectly to_date and to_timestamp).
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
import org.apache.spark.sql.functions.unix_timestamp val c1 = unix_timestamp() scala> c1.explain(true) unix_timestamp(current_timestamp(), yyyy-MM-dd HH:mm:ss, None) scala> println(c1.expr.numberedTreeString) 00 unix_timestamp(current_timestamp(), yyyy-MM-dd HH:mm:ss, None) 01 :- current_timestamp() 02 +- yyyy-MM-dd HH:mm:ss import org.apache.spark.sql.catalyst.expressions.UnixTimestamp scala> c1.expr.isInstanceOf[UnixTimestamp] res0: Boolean = true |
|
Note
|
UnixTimestamp is UnixTime expression internally (as is ToUnixTimestamp expression).
|
UnixTimestamp supports StringType, DateType and TimestampType as input types for a time expression and returns LongType.
|
1 2 3 4 5 6 |
scala> c1.expr.eval() res1: Any = 1493354303 |
UnixTimestamp uses DateTimeUtils.newDateFormat for date/time format (as Java’s java.text.DateFormat).
