hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yanbo Liang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3704) In the DFSClient, Add the node to the dead list when the ipc.Client calls fails
Date Wed, 24 Oct 2012 11:00:17 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483134#comment-13483134
] 

Yanbo Liang commented on HDFS-3704:
-----------------------------------

Add a specific DN to deadNodes means we don't take it as the first candidate when reading
a block. So when call the ipc interface to get the visible length of the file/last block,
the DN will get the number of bytes has ACKed by the downstream DN. But at the same time,
the generation stamp will be compared. If the replica's GS is older/smaller than the block's
GS which is assigned by the NN, it means this replica is not validate and throws IOException.
So here we can not just take IOException which isn't a ReplicaNotFoundException as the DN
is temporary or permanent unavailable. We should distinguish different causation of IOException
which throws during process this ipc call(getReplicaVisibleLength) at DN and name for different
Exception inherited from IOException. 
The failure in TestUnderReplicatedBlocks is related with HDFS-4067.
                
> In the DFSClient, Add the node to the dead list when the ipc.Client calls fails
> -------------------------------------------------------------------------------
>
>                 Key: HDFS-3704
>                 URL: https://issues.apache.org/jira/browse/HDFS-3704
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 1.0.3, 2.0.0-alpha
>            Reporter: nkeywal
>            Priority: Minor
>         Attachments: HADOOP-3704.patch
>
>
> The DFSCLient maintains a list of dead node per input steam. When creating this DFSInputStream,
it may connect to one of the nodes to check final block size. If this call fail, this datanode
should be put in the dead nodes list to save time. If not it will be retried for the block
transfer during the read, and we're likely to get a timeout.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message