hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8829) Make SO_RCVBUF and SO_SNDBUF size configurable for DataTransferProtocol sockets and allow configuring auto-tuning
Date Tue, 15 Sep 2015 02:00:50 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14744684#comment-14744684
] 

Hudson commented on HDFS-8829:
------------------------------

FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #385 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/385/])
HDFS-8829. Make SO_RCVBUF and SO_SNDBUF size configurable for DataTransferProtocol sockets
and allow configuring auto-tuning (He Tianyi via Colin P. McCabe) (cmccabe: rev 7b5cf5352efedc7d7ebdbb6b58f1b9a688812e75)
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiverServer.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/net/TcpPeerServer.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/net/PeerServer.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java
* hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeTransferSocketSize.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/net/DomainPeerServer.java


> Make SO_RCVBUF and SO_SNDBUF size configurable for DataTransferProtocol sockets and allow
configuring auto-tuning
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-8829
>                 URL: https://issues.apache.org/jira/browse/HDFS-8829
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>    Affects Versions: 2.3.0, 2.6.0
>            Reporter: He Tianyi
>            Assignee: He Tianyi
>             Fix For: 2.8.0
>
>         Attachments: HDFS-8829.0001.patch, HDFS-8829.0002.patch, HDFS-8829.0003.patch,
HDFS-8829.0004.patch, HDFS-8829.0005.patch, HDFS-8829.0006.patch
>
>
> {code:java}
>   private void initDataXceiver(Configuration conf) throws IOException {
>     // find free port or use privileged port provided
>     TcpPeerServer tcpPeerServer;
>     if (secureResources != null) {
>       tcpPeerServer = new TcpPeerServer(secureResources);
>     } else {
>       tcpPeerServer = new TcpPeerServer(dnConf.socketWriteTimeout,
>           DataNode.getStreamingAddr(conf));
>     }
>     tcpPeerServer.setReceiveBufferSize(HdfsConstants.DEFAULT_DATA_SOCKET_SIZE);
> {code}
> The last line sets SO_RCVBUF explicitly, thus disabling tcp auto-tuning on some system.
> Shall we make this behavior configurable?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message