hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "nkeywal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3704) In the DFSClient, Add the node to the dead list when the ipc.Client calls fails
Date Tue, 23 Oct 2012 16:41:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13482458#comment-13482458
] 

nkeywal commented on HDFS-3704:
-------------------------------

The dead note list exists already, but is used only when it's a read error, not a call to
the ipc interface to get the file length. The existing algo takes into account transient errors:
the dead nodes list is cleared if there is no remaining node available. However, today, as
we don't take into account the initial ipc call, we're doing two successive tries to the same
suspicious node instead of trying one of the others.

                
> In the DFSClient, Add the node to the dead list when the ipc.Client calls fails
> -------------------------------------------------------------------------------
>
>                 Key: HDFS-3704
>                 URL: https://issues.apache.org/jira/browse/HDFS-3704
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 1.0.3, 2.0.0-alpha
>            Reporter: nkeywal
>            Priority: Minor
>         Attachments: HADOOP-3704.patch, HADOOP-3704.patch
>
>
> The DFSCLient maintains a list of dead node per input steam. When creating this DFSInputStream,
it may connect to one of the nodes to check final block size. If this call fail, this datanode
should be put in the dead nodes list to save time. If not it will be retried for the block
transfer during the read, and we're likely to get a timeout.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message