RelationProvider Contract — Relation Providers With Schema Inference
RelationProvider
is the contract of BaseRelation providers that create a relation with schema inference.
Note
|
Schema inference is also called schema discovery. |
The requirement of not specifying a user-defined schema or having one that does not match the relation is enforced when DataSource
is requested for a BaseRelation for a given data source format. If specified and does not match, DataSource
throws a AnalysisException
:
1 2 3 4 5 |
[className] does not allow user-specified schemas. |
1 2 3 4 5 6 7 8 9 10 11 |
package org.apache.spark.sql.sources trait RelationProvider { def createRelation( sqlContext: SQLContext, parameters: Map[String, String]): BaseRelation } |
Method | Description |
---|---|
|
Creates a BaseRelation for loading data from an external data source Used exclusively when |
RelationProvider | Description |
---|---|
Tip
|
Use SchemaRelationProvider for relation providers that require a user-defined schema. |