hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gopal V (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11226) ipc.Client has to use setTrafficClass() with IPTOS_LOWDELAY|IPTOS_RELIABILITY
Date Fri, 23 Jan 2015 22:00:41 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14290067#comment-14290067
] 

Gopal V commented on HADOOP-11226:
----------------------------------

Thanks [~cnauroth]. I think this needs testing beyond what I've done - my test-setup is a
single rack with has 10G ethernet - using both H-TCP & Cubic congestion controls.

This definitely needs more testing, should be even better when encountering cross-rack packet
loss.

In particular, I'd love to see this patch tested with JDK7 Infiniband native support. 

Hadoop IPC/DataStreamer/Shuffle can already use Infiniband zero-copy mode by adding JVM opt
args configuration options {{-Dcom.sun.sdp.conf=/etc/sdp.conf -Dcom.sun.sdp.debug=/tmp/sdp-debug.log}},
provided the OS/Hardware has support.

> ipc.Client has to use setTrafficClass() with IPTOS_LOWDELAY|IPTOS_RELIABILITY
> -----------------------------------------------------------------------------
>
>                 Key: HADOOP-11226
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11226
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: ipc
>    Affects Versions: 2.6.0
>            Reporter: Gopal V
>            Assignee: Gopal V
>              Labels: Infiniband
>         Attachments: HADOOP-11226.1.patch, HADOOP-11226.2.patch
>
>
> During heavy shuffle, packet loss for IPC packets was observed from a machine.
> Avoid packet-loss and speed up transfer by using 0x14 QOS bits for the packets.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message