SchemaRelationProvider Contract — Relation Providers With Mandatory User-Defined Schema
SchemaRelationProvider
is the contract of BaseRelation providers that require a user-defined schema while creating a relation.
The requirement of specifying a user-defined schema is enforced when DataSource
is requested for a BaseRelation for a given data source format. If not specified, DataSource
throws a AnalysisException
:
1 2 3 4 5 |
A schema needs to be specified when using [className]. |
1 2 3 4 5 6 7 8 9 10 11 12 |
package org.apache.spark.sql.sources trait SchemaRelationProvider { def createRelation( sqlContext: SQLContext, parameters: Map[String, String], schema: StructType): BaseRelation } |
Method | Description |
---|---|
|
Creates a BaseRelation for the user-defined schema Used exclusively when |
Note
|
There are no known direct implementation of PrunedFilteredScan Contract in Spark SQL. |
Tip
|
Use RelationProvider for data source providers with schema inference. |
Tip
|
Use both SchemaRelationProvider and RelationProvider if a data source should support both schema inference and user-defined schemas.
|