hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-3416) DFS write pipeline : DFSClient sometimes does not detect second datanode failure
Date Mon, 19 May 2008 18:09:56 GMT
DFS write pipeline : DFSClient sometimes does not detect second datanode failure 
---------------------------------------------------------------------------------

                 Key: HADOOP-3416
                 URL: https://issues.apache.org/jira/browse/HADOOP-3416
             Project: Hadoop Core
          Issue Type: Bug
    Affects Versions: 0.16.0
            Reporter: Raghu Angadi




When the first datanode's write to second datanode fails or times out DFSClient ends up marking
first datanode as the bad one and removes it from the pipeline. Similar problem exists on
DataNode as well and it is fixed in HADOOP-3339. From HADOOP-3339 : 

"The main issue is that BlockReceiver thread (and DataStreamer in the case of DFSClient) interrupt()
the 'responder' thread. But interrupting is a pretty coarse control. We don't know what state
the responder is in and interrupting has different effects depending on responder state. To
fix this properly we need to redesign how we handle these interactions."

When the first datanode closes its socket from DFSClient, DFSClient should properly read all
the data left in the socket.. Also, DataNode's closing of the socket should not result in
a TCP reset, otherwise I think DFSClient will not be able to read from the socket.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message