TableScan Contract — Relations with Column Pruning

2012-06-18 分类：spark-sql 阅读(1283) 评论(0)

TableScan Contract — Relations with Column Pruning

TableScan is the contract of BaseRelations with support for column pruning, i.e. can eliminate unneeded columns before producing an RDD containing all of its tuples as Row objects.



package org.apache.spark.sql.sources

trait PrunedScan {
  def buildScan(): RDD[Row]
}

package org.apache.spark.sql.sources

trait PrunedScan {

def buildScan(): RDD[Row]

}

Table 1. TableScan Contract
Property	Description
`buildScan`	Building distributed data scan with column pruning In other words, `buildScan` creates a `RDD[Row]` to represent a distributed data scan (i.e. scanning over data in a relation). Used exclusively when `DataSourceStrategy` execution planning strategy is requested to plan a LogicalRelation with a TableScan.