SpecificParquetRecordReaderBase-spark技术分享

SpecificParquetRecordReaderBase — Hadoop RecordReader

SpecificParquetRecordReaderBase is the base Hadoop RecordReader for parquet format readers that directly materialize to T.

Note	RecordReader reads `<key, value>` pairs from an Hadoop `InputSplit`.

Note	VectorizedParquetRecordReader is the one and only `SpecificParquetRecordReaderBase` that directly materialize to Java `Objects`.

Table 1. SpecificParquetRecordReaderBase’s Internal Properties (e.g. Registries, Counters and Flags)
Name	Description
`sparkSchema`	Spark schema Initialized when `SpecificParquetRecordReaderBase` is requested to initialize (from the value of org.apache.spark.sql.parquet.row.requested_schema configuration as set when `ParquetFileFormat` is requested to build a data reader with partition column values appended)



void initialize(InputSplit inputSplit, TaskAttemptContext taskAttemptContext)

void initialize(InputSplit inputSplit, TaskAttemptContext taskAttemptContext)

Note	`initialize` is part of RecordReader Contract to initialize a `RecordReader`.

initialize…FIXME