hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9700) DFSClient and DFSOutputStream do not respect TCP_NODELAY config in two spots
Date Tue, 02 Feb 2016 20:32:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15128971#comment-15128971

Colin Patrick McCabe commented on HDFS-9700:

Good find, [~ghelmling].

bq. [~iwasakims] wrote: Adding configuration key such as HdfsClientConfigKeys.DFS_CLIENT_SOCKET_TCP_NODELAY
might be conservative option to retain existing behaviour and change the default value later.
(You can see HDFS-8829 and HDFS-9259 as example for the fix.)

Yeah, it makes sense to have a separate configuration key controlling whether {{TCP_NODELAY}}
is set on {{DataTransferProtocol}}.

I think we should change the default to be that TCP_NODELAY is "on" for both {{DataTransferProtocol}}
and Hadoop RPC.  We already try to avoid sending small messages over DataTransferProtocol,
so Nagle's algorithm doesn't add a lot (and may significantly degrade the performance of things
like hflush and hsync).

> DFSClient and DFSOutputStream do not respect TCP_NODELAY config in two spots
> ----------------------------------------------------------------------------
>                 Key: HDFS-9700
>                 URL: https://issues.apache.org/jira/browse/HDFS-9700
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>    Affects Versions: 2.7.1, 2.6.3
>            Reporter: Gary Helmling
>            Assignee: Gary Helmling
>         Attachments: HDFS-9700-v1.patch, HDFS-9700_branch-2.7.patch
> In {{DFSClient.connectToDN()}} and {{DFSOutputStream.createSocketForPipeline()}}, we
never call {{setTcpNoDelay()}} on the constructed socket before sending.  In both cases, we
should respect the value of ipc.client.tcpnodelay in the configuration.
> While this applies whether security is enabled or not, it seems to have a bigger impact
on latency when security is enabled.

This message was sent by Atlassian JIRA

View raw message