hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1567) DFSClient should retry reading from all datanodes in round robin fashion
Date Tue, 04 Jan 2011 06:47:45 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977145#action_12977145

dhruba borthakur commented on HDFS-1567:

Just to elaborate, the above is applicable only for *retries* and retries occur when the client
encounters errors while reading a block from a datanode.

> DFSClient should retry reading from all datanodes in round robin fashion
> ------------------------------------------------------------------------
>                 Key: HDFS-1567
>                 URL: https://issues.apache.org/jira/browse/HDFS-1567
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs client
>            Reporter: dhruba borthakur
> In the current implementation, the DFSClient retries the same datanode a few times (for
reading) before marking the datanode as "dead" and moving on to trying the read-request from
the next replica. This introduces additional delay when the first candidate datanode was really
out-of-service. Instead, it might be beneficial (for performance) to make the DFSClient retry
all datanode replicas one after another (while keeping the total number of retries per datanode
the same as now); if all the replicas were tried unsuccessfully once, then loop back to retrying
the first one and so on.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message