当前位置：spark技术分享 > spark-sql > 正文

ColumnarBatch

2012-04-08 分类：spark-sql 阅读(1469) 评论(0)

ColumnarBatch

ColumnarBatch is…FIXME

ColumnarBatch is created when:

InMemoryTableScanExec is requested to createAndDecompressColumn
VectorizedParquetRecordReader is requested to initBatch
OrcColumnarBatchReader is requested to initBatch
ColumnVectorUtils is requested to toBatch
ArrowPythonRunner is requested for a Iterator[ColumnarBatch] (i.e. newReaderIterator)
ArrowConverters is requested for a ArrowRowIterator (i.e. fromPayloadIterator)

Note	`ColumnarBatch` is an `Evolving` contract that is evolving towards becoming a stable API, but is not a stable API yet and can change from one feature release to another release. In other words, using the contract is as treading on thin ice.

ColumnarBatch takes an array of ColumnVectors when created. ColumnarBatch initializes the internal MutableColumnarRow.

Table 1. ColumnarBatch’s Internal Properties (e.g. Registries, Counters and Flags)
Name	Description
`numRows`	Number of rows
`row`	`MutableColumnarRow`

赞(0) 打赏

未经允许不得转载：spark技术分享 » ColumnarBatch

标签：spark-sql

相关推荐

关注公众号：spark技术分享

联系我们联系我们

QQ咨询
QQ咨询
回顶
回顶部