TransportRequestHandler-spark技术分享

TransportRequestHandler

TransportRequestHandler is a MessageHandler of RequestMessage messages from Netty’s Channel.

TransportRequestHandler is created exclusively when TransportContext is requested to createChannelHandler.

Tip

Enable ERROR, WARN, TRACE logging level for org.apache.spark.network.server.TransportRequestHandler logger to see what happens inside.

Add the following line to conf/log4j.properties:



log4j.logger.org.apache.spark.network.server.TransportRequestHandler=TRACE

log4j.logger.org.apache.spark.network.server.TransportRequestHandler=TRACE

Refer to Logging.

`processRpcRequest` Internal Method



void processRpcRequest(final RpcRequest req)

void processRpcRequest(final RpcRequest req)

processRpcRequest…FIXME

Note	`processRpcRequest` is used exclusively when `TransportRequestHandler` is requested to handle a `RpcRequest` request.

`processFetchRequest` Internal Method



void processFetchRequest(final ChunkFetchRequest req)

void processFetchRequest(final ChunkFetchRequest req)

processFetchRequest…FIXME

Note	`processFetchRequest` is used exclusively when `TransportRequestHandler` is requested to handle a `ChunkFetchRequest` request.

`processOneWayMessage` Internal Method



void processOneWayMessage(OneWayMessage req)

void processOneWayMessage(OneWayMessage req)

processOneWayMessage…FIXME

Note	`processOneWayMessage` is used exclusively when `TransportRequestHandler` is requested to handle a `OneWayMessage` request.

`processStreamRequest` Internal Method



void processStreamRequest(final StreamRequest req)

void processStreamRequest(final StreamRequest req)

processStreamRequest…FIXME

Note	`processStreamRequest` is used exclusively when `TransportRequestHandler` is requested to handle a `StreamRequest` request.

Handling RequestMessages — `handle` Method



void handle(RequestMessage request)

void handle(RequestMessage request)

Note	`handle` is part of MessageHandler Contract to handle messages (aka requests).

handle branches off per the type of the input RequestMessage:

For ChunkFetchRequest requests, handle processFetchRequest
For RpcRequest requests, handle processRpcRequest
For OneWayMessage requests, handle processOneWayMessage
For StreamRequest requests, handle processStreamRequest

For unknown requests, handle simply throws a IllegalArgumentException.



Unknown request type: [request]

Unknown request type: [request]

Creating TransportRequestHandler Instance

TransportRequestHandler takes the following when created:

Netty’s Channel
TransportClient
RpcHandler
Maximum number of chunks allowed to be transferred at the same time

TransportRequestHandler initializes the internal registries and counters.

TransportRequestHandler