spark技术分享KafkaRelation
KafkaRelation is…FIXME
KafkaRelation is created when…FIXME
Creating KafkaRelation Instance
KafkaRelation takes the following when created:
KafkaRelation initializes the internal registries and counters.
getPartitionOffsets Internal Method
|
1 2 3 4 5 6 7 |
getPartitionOffsets( kafkaReader: KafkaOffsetReader, kafkaOffsets: KafkaOffsetRangeLimit): Map[TopicPartition, Long] |
|
Caution
|
FIXME |
|
Note
|
getPartitionOffsets is used exclusively when KafkaRelation builds RDD of rows (from the tuples).
|
