spark技术分享HiveMetastoreCatalog — Legacy SessionCatalog for Converting Hive Metastore Relations to Data Source Relations
HiveMetastoreCatalog is a legacy session-scoped catalog of relational entities that HiveSessionCatalog uses exclusively for converting Hive metastore relations to data source relations (when RelationConversions logical evaluation rule is executed).
HiveMetastoreCatalog is created exclusively when HiveSessionStateBuilder is requested for SessionCatalog (and creates a HiveSessionCatalog).
Figure 1. HiveMetastoreCatalog, HiveSessionCatalog and HiveSessionStateBuilder
HiveMetastoreCatalog takes a SparkSession when created.
Converting HiveTableRelation to LogicalRelation — convertToLogicalRelation Method
|
1 2 3 4 5 6 7 8 9 |
convertToLogicalRelation( relation: HiveTableRelation, options: Map[String, String], fileFormatClass: Class[_ <: FileFormat], fileType: String): LogicalRelation |
convertToLogicalRelation…FIXME
|
Note
|
convertToLogicalRelation is used exclusively when RelationConversions logical evaluation rule is requested to convert a HiveTableRelation to a LogicalRelation for parquet, native and hive ORC storage formats.
|
inferIfNeeded Internal Method
|
1 2 3 4 5 6 7 8 9 |
inferIfNeeded( relation: HiveTableRelation, options: Map[String, String], fileFormat: FileFormat, fileIndexOpt: Option[FileIndex] = None): CatalogTable |
inferIfNeeded…FIXME
|
Note
|
inferIfNeeded is used exclusively when HiveMetastoreCatalog is requested to convertToLogicalRelation.
|
