CreatableRelationProvider Contract — Data Sources That Write Rows Per Save Mode-spark技术分享

CreatableRelationProvider Contract — Data Sources That Write Rows Per Save Mode

CreatableRelationProvider is the contract for data source providers that want to support writing the rows of a structured query (a DataFrame) per save mode.



package org.apache.spark.sql.sources

trait CreatableRelationProvider {
  def createRelation(
    sqlContext: SQLContext,
    mode: SaveMode,
    parameters: Map[String, String],
    data: DataFrame): BaseRelation
}

package org.apache.spark.sql.sources

trait CreatableRelationProvider {

def createRelation(

sqlContext: SQLContext,

mode: SaveMode,

parameters: Map[String, String],

data: DataFrame): BaseRelation

}

Table 1. CreatableRelationProvider Contract
Method	Description
`createRelation`	Saving the rows of a structured query (a DataFrame) to an external data source `createRelation` saves the rows of a structured query (a DataFrame) to a target relation per save mode and parameters. In the end, `createRelation` creates a BaseRelation to represent the relation created. The save mode specifies what happens when the destination already exists and can be one of the following: `Append` `ErrorIfExists` `Ignore` `Overwrite`

CreatableRelationProvider is used when:

DataSource is requested to write the result of a structured query to data source per save mode (after DataFrameWriter is requested to save)
DataSource is requested to write the result of a structured query to data source per save mode followed by reading rows back (after DataFrameWriter is requested to save to a non-Hive table or for Create Table As Select (CTAS) SQL statements)

Table 2. CreatableRelationProviders
CreatableRelationProvider	Description
JdbcRelationProvider	Data source provider for JDBC data source
KafkaSourceProvider	Data source provider for Kafka data source

CreatableRelationProvider Contract — Data Sources That Write Rows Per Save Mode

CreatableRelationProvider Contract — Data Sources That Write Rows Per Save Mode

相关推荐

欢迎关注：spark技术分享

热门标签

近期文章

分类目录

关注公众号：spark技术分享

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

微信扫一扫打赏

QQ咨询

回顶部