GlobalTempViewManager — Management Interface of Global Temporary Views-spark技术分享

GlobalTempViewManager — Management Interface of Global Temporary Views

GlobalTempViewManager is the interface to manage global temporary views (that SessionCatalog uses when requested to create, alter or drop global temporary views).

Strictly speaking, GlobalTempViewManager simply manages the names of the global temporary views registered (and the corresponding logical plans) and has no interaction with other services in Spark SQL.

GlobalTempViewManager is available as globalTempViewManager property of a SharedState.

Figure 1. GlobalTempViewManager and SparkSession



scala> :type spark
org.apache.spark.sql.SparkSession

scala> :type spark.sharedState.globalTempViewManager
org.apache.spark.sql.catalyst.catalog.GlobalTempViewManager

scala> :type spark

org.apache.spark.sql.SparkSession

scala> :type spark.sharedState.globalTempViewManager

org.apache.spark.sql.catalyst.catalog.GlobalTempViewManager

Table 1. GlobalTempViewManager API

Method

Description

clear



clear(): Unit

clear(): Unit

create



create(
  name: String,
  viewDefinition: LogicalPlan,
  overrideIfExists: Boolean): Unit

create(

name: String,

viewDefinition: LogicalPlan,

overrideIfExists: Boolean): Unit

get



get(name: String): Option[LogicalPlan]

get(name: String): Option[LogicalPlan]

listViewNames



listViewNames(pattern: String): Seq[String]

listViewNames(pattern: String): Seq[String]

remove



remove(name: String): Boolean

remove(name: String): Boolean

rename



rename(oldName: String, newName: String): Boolean

rename(oldName: String, newName: String): Boolean

update



update(name: String, viewDefinition: LogicalPlan): Boolean

update(name: String, viewDefinition: LogicalPlan): Boolean

GlobalTempViewManager is created exclusively when SharedState is requested for one (for the very first time only as it is cached).

GlobalTempViewManager takes the name of the database when created.

spark sql GlobalTempViewManager creating instance.png

Figure 2. Creating GlobalTempViewManager

Table 2. GlobalTempViewManager’s Internal Properties (e.g. Registries, Counters and Flags)
Name	Description
`viewDefinitions`	Registry of global temporary view definitions as logical plans per view name.

`clear` Method



clear(): Unit

clear(): Unit

clear simply removes all the entries in the viewDefinitions internal registry.

Note	`clear` is used when `SessionCatalog` is requested to reset (that happens to be exclusively in the Spark SQL internal tests).

Creating (Registering) Global Temporary View (Definition) — `create` Method



create(
  name: String,
  viewDefinition: LogicalPlan,
  overrideIfExists: Boolean): Unit

create(

name: String,

viewDefinition: LogicalPlan,

overrideIfExists: Boolean): Unit

create simply registers (adds) the input LogicalPlan under the input name.

create throws an AnalysisException when the input overrideIfExists flag is off and the viewDefinitions internal registry contains the input name.



Temporary view '[table]' already exists

Temporary view '[table]' already exists

Note	`create` is used when `SessionCatalog` is requested to createGlobalTempView (when CreateViewCommand and CreateTempViewUsing logical commands are executed).

Retrieving Global View Definition Per Name — `get` Method



get(name: String): Option[LogicalPlan]

get(name: String): Option[LogicalPlan]

get simply returns the LogicalPlan that was registered under the name if it defined.

Note	`get` is used when `SessionCatalog` is requested to getGlobalTempView, getTempViewOrPermanentTableMetadata, lookupRelation, isTemporaryTable or refreshTable.

Listing Global Temporary Views For Pattern — `listViewNames` Method



listViewNames(pattern: String): Seq[String]

listViewNames(pattern: String): Seq[String]

listViewNames simply gives a list of the global temporary views with names matching the input pattern.

Note	`listViewNames` is used exclusively when `SessionCatalog` is requested to listTables

Removing (De-Registering) Global Temporary View — `remove` Method



remove(name: String): Boolean

remove(name: String): Boolean

remove simply tries to remove the name from the viewDefinitions internal registry and returns true when removed or false otherwise.

Note	`remove` is used when `SessionCatalog` is requested to drop a global temporary view or table.

`rename` Method



rename(oldName: String, newName: String): Boolean

rename(oldName: String, newName: String): Boolean

rename…FIXME

Note	`rename` is used when…FIXME

`update` Method



update(name: String, viewDefinition: LogicalPlan): Boolean

update(name: String, viewDefinition: LogicalPlan): Boolean

update…FIXME

Note	`update` is used exclusively when `SessionCatalog` is requested to alter a global temporary view.

GlobalTempViewManager — Management Interface of Global Temporary Views