hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7557) Make IPC and RPC headers be extensible
Date Thu, 08 Sep 2011 17:57:10 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100506#comment-13100506

Todd Lipcon commented on HADOOP-7557:

Yea, I guess PBs also have type information. But type information means the value can be much
shorter as well.

bq. I also question whether the size of RPC headers is a significant factor in Hadoop performance.
Even when the payload is only a few bytes, the dominant cost is not network bandwidth nor
request parsing.

In Hadoop job performance, I agree. In something like HBase, though, it is a significant amount
of the time spent when requests are doing small reads out of cache. Currently HBase has a
fork of Hadoop IPC, but it would be good to keep that use case in mind in hopes that down
the road, we can converge implementations again.

It seems the easiest way to settle the performance debate is to do a benchmark?

> Make  IPC and RPC headers be extensible
> ---------------------------------------
>                 Key: HADOOP-7557
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7557
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Sanjay Radia

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message