ParseToDate-spark技术分享

ParseToDate Expression

ParseToDate is a RuntimeReplaceable expression that represents the to_date function (in logical query plans).



// DEMO to_date(e: Column): Column
// DEMO to_date(e: Column, fmt: String): Column

// DEMO to_date(e: Column): Column

// DEMO to_date(e: Column, fmt: String): Column

As a RuntimeReplaceable expression, ParseToDate is replaced by Catalyst Optimizer with the child expression:

Cast(left, DateType) for to_date(e: Column): Column function
Cast(Cast(UnixTimestamp(left, format), TimestampType), DateType) for to_date(e: Column, fmt: String): Column function



// FIXME DEMO Conversion to `Cast(left, DateType)`
// FIXME DEMO Conversion to `Cast(Cast(UnixTimestamp(left, format), TimestampType), DateType)`

// FIXME DEMO Conversion to `Cast(left, DateType)`

// FIXME DEMO Conversion to `Cast(Cast(UnixTimestamp(left, format), TimestampType), DateType)`

ParseToDate takes the following when created: