hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7523) Setting a socket receive buffer size in DFSClient
Date Tue, 30 Dec 2014 17:50:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14261299#comment-14261299
] 

stack commented on HDFS-7523:
-----------------------------

[~xieliang007]

bq. let me report number once get change

Before I commit here?

bq. ...and i think we also need to make the buffer size configurable.

Makes sense.  You want to do this in new issue?

> Setting a socket receive buffer size in DFSClient
> -------------------------------------------------
>
>                 Key: HDFS-7523
>                 URL: https://issues.apache.org/jira/browse/HDFS-7523
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: dfsclient
>    Affects Versions: 2.6.0
>            Reporter: Liang Xie
>            Assignee: Liang Xie
>         Attachments: HDFS-7523-001 (1).txt, HDFS-7523-001 (1).txt, HDFS-7523-001.txt,
HDFS-7523-001.txt, HDFS-7523-001.txt
>
>
> It would be nice if we have a socket receive buffer size while creating socket from client(HBase)
view, in old version it should be in DFSInputStream, in trunk it seems should be at:
> {code}
>   @Override // RemotePeerFactory
>   public Peer newConnectedPeer(InetSocketAddress addr,
>       Token<BlockTokenIdentifier> blockToken, DatanodeID datanodeId)
>       throws IOException {
>     Peer peer = null;
>     boolean success = false;
>     Socket sock = null;
>     try {
>       sock = socketFactory.createSocket();
>       NetUtils.connect(sock, addr,
>         getRandomLocalInterfaceAddr(),
>         dfsClientConf.socketTimeout);
>       peer = TcpPeerServer.peerFromSocketAndKey(saslClient, sock, this,
>           blockToken, datanodeId);
>       peer.setReadTimeout(dfsClientConf.socketTimeout);
> {code}
> e.g: sock.setReceiveBufferSize(HdfsConstants.DEFAULT_DATA_SOCKET_SIZE);
> the default socket buffer size in Linux+JDK7 seems is 8k if i am not wrong, this value
sometimes is small for HBase 64k block reading in a 10G network(at least, more system call)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message