hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8069) Enable TCP_NODELAY by default for IPC
Date Thu, 16 Feb 2012 06:45:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209150#comment-13209150
] 

Todd Lipcon commented on HADOOP-8069:
-------------------------------------

My hunch is that it's pretty small. I think the only RPC to the NN which would be at all frequent
and cross the 8K boundary would be getListing(). On one production hbase cluster I collected
metrics from a while back, getListing represented 8.3% of the RPCs. On one of our QA clusters
that's been running MR workloads, it represents 2.3%. Unfortunately we don't have enough metrics
to get any info on the size distribution of those responses.

Would be interested to hear if some of your production clusters show a similar mix.
                
> Enable TCP_NODELAY by default for IPC
> -------------------------------------
>
>                 Key: HADOOP-8069
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8069
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: ipc
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hadoop-8069.txt
>
>
> I think we should switch the default for the IPC client and server NODELAY options to
true. As wikipedia says:
> {quote}
> In general, since Nagle's algorithm is only a defense against careless applications,
it will not benefit a carefully written application that takes proper care of buffering; the
algorithm has either no effect, or negative effect on the application.
> {quote}
> Since our IPC layer is well contained and does its own buffering, we shouldn't be careless.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message