hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8069) Enable TCP_NODELAY by default for IPC
Date Thu, 16 Feb 2012 00:04:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208994#comment-13208994
] 

Daryn Sharp commented on HADOOP-8069:
-------------------------------------

The short packets should result 40 byte packet overhead.  Delayed ack should coalesce the
ack to prevent any overhead there.  Overall, the overhead for short packets should be ~0.5%.

The key is avoiding 200ms bubbles in the default settings for a socket.  From memory: nagle
holds data until a full packet is assembled, it receives an ack for the previous packet, or
200ms expires.  Receiver sends delayed acks for every other packet, or 200ms expires.  If
the last partial packet is an odd packet, the sender and receiver are waiting for each other
to send something.  The receiver's 200ms dack timer expires, it sends the ack for the next
to last even packet, sender sends the last odd packet.

I _think_, but I'm rusty, that the main differences between nagle and cork are:
* nagle may send a partial packet if the receiver acks before another full packet is assembled
* cork ignores acks and just sends full packets, or 200ms expires
* uncorking flushes the socket buffer and sets the tcp push flag (causing immediate ack, not
dack) on the partial packet 
* nodelay might be setting the push flag on all packets (generating 2X acks), but I think
it's just the partial packets
* nodelay is portable, cork is not

All said, I think #4 is probably the best bet.  It should in effect be like cork unless the
writes for a given chunk of data are written in a slow/sporadic fashion, thus causing acks
to send out partial packets.  Most comparisons are straight nodelay or straight cork, so your
findings will be interesting.
                
> Enable TCP_NODELAY by default for IPC
> -------------------------------------
>
>                 Key: HADOOP-8069
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8069
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: ipc
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hadoop-8069.txt
>
>
> I think we should switch the default for the IPC client and server NODELAY options to
true. As wikipedia says:
> {quote}
> In general, since Nagle's algorithm is only a defense against careless applications,
it will not benefit a carefully written application that takes proper care of buffering; the
algorithm has either no effect, or negative effect on the application.
> {quote}
> Since our IPC layer is well contained and does its own buffering, we shouldn't be careless.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message