UnresolvedRelation-spark技术分享

UnresolvedRelation Leaf Logical Operator for Table Reference

UnresolvedRelation is a leaf logical operator to represent a table reference in a logical query plan that has yet to be resolved (i.e. looked up in a catalog).

Note

If after Analyzer has finished analyzing a logical query plan the plan has still a UnresolvedRelation it fails the analyze phase with the following AnalysisException:



Table or view not found: [tableIdentifier]

Table or view not found: [tableIdentifier]

UnresolvedRelation is created when:

SparkSession is requested to create a DataFrame from a table
DataFrameWriter is requested to insert a DataFrame into a table
INSERT INTO (TABLE) or INSERT OVERWRITE TABLE SQL commands are executed
CreateHiveTableAsSelectCommand command is executed

Tip

Use table operator from Catalyst DSL to create a UnresolvedRelation logical operator, e.g. for testing or Spark SQL internals exploration.



import org.apache.spark.sql.catalyst.dsl.plans._
val plan = table(db = "myDB", ref = "t1")
scala> println(plan.numberedTreeString)
00 'UnresolvedRelation `myDB`.`t1`

import org.apache.spark.sql.catalyst.dsl.plans._

val plan = table(db = "myDB", ref = "t1")

scala> println(plan.numberedTreeString)

00 'UnresolvedRelation `myDB`.`t1`

Note	`UnresolvedRelation` is resolved to…FIXME

UnresolvedRelation

UnresolvedRelation Leaf Logical Operator for Table Reference

相关推荐

欢迎关注：spark技术分享

热门标签

近期文章

分类目录

关注公众号：spark技术分享

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

微信扫一扫打赏

QQ咨询

回顶部