hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-7265) Use a throttler for replica write in datanode
Date Sun, 19 Oct 2014 01:39:34 GMT
Tsz Wo Nicholas Sze created HDFS-7265:

             Summary: Use a throttler for replica write in datanode
                 Key: HDFS-7265
                 URL: https://issues.apache.org/jira/browse/HDFS-7265
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: datanode
            Reporter: Tsz Wo Nicholas Sze
            Assignee: Tsz Wo Nicholas Sze

BlockReceiver process packets in BlockReceiver.receivePacket() as follows
# read from socket
# enqueue the ack
# write to downstream
# write to disk

The above steps is repeated for each packet in a single thread.  When there are a lot of concurrent
writes in a datanode, the write time in #4 becomes very long.  As a result, it leads to SocketTimeoutException
since it cannot read from the socket for a long time.

This message was sent by Atlassian JIRA

View raw message