CatalogUtils Helper Object-spark技术分享

CatalogUtils Helper Object

CatalogUtils is a Scala object with the methods to support PreprocessTableCreation post-hoc logical resolution rule (among others).

Name Description

maskCredentials



maskCredentials(options: Map[String, String]): Map[String, String]

maskCredentials(options: Map[String, String]): Map[String, String]

Used when:

CatalogStorageFormat is requested to convert the storage specification to a LinkedHashMap
CreateTempViewUsing logical command is requested for the argString

normalizeBucketSpec



normalizeBucketSpec(
  tableName: String,
  tableCols: Seq[String],
  bucketSpec: BucketSpec,
  resolver: Resolver): BucketSpec

normalizeBucketSpec(

tableName: String,

tableCols: Seq[String],

bucketSpec: BucketSpec,

resolver: Resolver): BucketSpec

Used exclusively when PreprocessTableCreation post-hoc logical resolution rule is executed.

normalizePartCols



normalizePartCols(
  tableName: String,
  tableCols: Seq[String],
  partCols: Seq[String],
  resolver: Resolver): Seq[String]

normalizePartCols(

tableName: String,

tableCols: Seq[String],

partCols: Seq[String],

resolver: Resolver): Seq[String]

Used exclusively when PreprocessTableCreation post-hoc logical resolution rule is executed.

`normalizeColumnName` Internal Method



normalizeColumnName(
  tableName: String,
  tableCols: Seq[String],
  colName: String,
  colType: String,
  resolver: Resolver): String

normalizeColumnName(

tableName: String,

tableCols: Seq[String],

colName: String,

colType: String,

resolver: Resolver): String

normalizeColumnName…FIXME

Note	`normalizeColumnName` is used when `CatalogUtils` is requested to normalizePartCols and normalizeBucketSpec.

CatalogUtils Helper Object