关注 spark技术分享,
撸spark源码 玩spark最佳实践

RelationProvider Contract — Relation Providers With Schema Inference

RelationProvider Contract — Relation Providers With Schema Inference

Note
Schema inference is also called schema discovery.

The requirement of not specifying a user-defined schema or having one that does not match the relation is enforced when DataSource is requested for a BaseRelation for a given data source format. If specified and does not match, DataSource throws a AnalysisException:

Table 1. RelationProvider Contract
Method Description

createRelation

Creates a BaseRelation for loading data from an external data source

Used exclusively when DataSource is requested for a BaseRelation for a given data source format (and no user-defined schema or the user-defined schema matches schema of the BaseRelation)

Table 2. RelationProviders
RelationProvider Description

JdbcRelationProvider

KafkaSourceProvider

Tip
Use SchemaRelationProvider for relation providers that require a user-defined schema.
赞(0) 打赏
未经允许不得转载:spark技术分享 » RelationProvider Contract — Relation Providers With Schema Inference
分享到: 更多 (0)

关注公众号:spark技术分享

联系我们联系我们

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

微信扫一扫打赏