hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-7523) Setting a socket receive buffer size in DFSClient
Date Mon, 22 Dec 2014 17:13:13 GMT

     [ https://issues.apache.org/jira/browse/HDFS-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HDFS-7523:
------------------------
    Attachment: HDFS-7523-001 (1).txt

Retry.  Looking at recent hadoopqa history, there's a few flakies and TestSymlinkHdfsFileContext
is failing near always.

> Setting a socket receive buffer size in DFSClient
> -------------------------------------------------
>
>                 Key: HDFS-7523
>                 URL: https://issues.apache.org/jira/browse/HDFS-7523
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: dfsclient
>    Affects Versions: 2.6.0
>            Reporter: Liang Xie
>            Assignee: Liang Xie
>         Attachments: HDFS-7523-001 (1).txt, HDFS-7523-001.txt, HDFS-7523-001.txt, HDFS-7523-001.txt
>
>
> It would be nice if we have a socket receive buffer size while creating socket from client(HBase)
view, in old version it should be in DFSInputStream, in trunk it seems should be at:
> {code}
>   @Override // RemotePeerFactory
>   public Peer newConnectedPeer(InetSocketAddress addr,
>       Token<BlockTokenIdentifier> blockToken, DatanodeID datanodeId)
>       throws IOException {
>     Peer peer = null;
>     boolean success = false;
>     Socket sock = null;
>     try {
>       sock = socketFactory.createSocket();
>       NetUtils.connect(sock, addr,
>         getRandomLocalInterfaceAddr(),
>         dfsClientConf.socketTimeout);
>       peer = TcpPeerServer.peerFromSocketAndKey(saslClient, sock, this,
>           blockToken, datanodeId);
>       peer.setReadTimeout(dfsClientConf.socketTimeout);
> {code}
> e.g: sock.setReceiveBufferSize(HdfsConstants.DEFAULT_DATA_SOCKET_SIZE);
> the default socket buffer size in Linux+JDK7 seems is 8k if i am not wrong, this value
sometimes is small for HBase 64k block reading in a 10G network(at least, more system call)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message