hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-1850) DN should transmit absolute failed volume count rather than increments to the NN
Date Mon, 02 May 2011 23:16:04 GMT

     [ https://issues.apache.org/jira/browse/HDFS-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Eli Collins updated HDFS-1850:

    Attachment: hdfs-1850-7.patch

TestDatanodeBlockScanner passes for me in eclipse and when looped from the command line. I
think this is related to an earlier change. The error message indicates we need to bump the
number of attempts (ie it did see 1 corrupt replica after 20 attempts, but it also bails on
the 20th attempt).

java.util.concurrent.TimeoutException: Timed out waiting for corrupt replicas. Waiting for
1, but only found 1

I'll bump the # attempts to 50 so we're more tolerant.

Updated patch attached. 

> DN should transmit absolute failed volume count rather than increments to the NN
> --------------------------------------------------------------------------------
>                 Key: HDFS-1850
>                 URL: https://issues.apache.org/jira/browse/HDFS-1850
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: data-node, name-node
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>             Fix For: 0.22.0
>         Attachments: hdfs-1850-1.patch, hdfs-1850-2.patch, hdfs-1850-3.patch, hdfs-1850-4.patch,
hdfs-1850-5.patch, hdfs-1850-6.patch, hdfs-1850-7.patch
> The API added in HDFS-811 for the DN to report volume failures to the NN is "inc(DN)".
However the given sequence of events will result in the NN forgetting about reported failed
> # DN loses a volume and reports it
> # NN restarts
> # DN re-registers to the new NN
> A more robust interface would be to have the DN report the total number of volume failures
to the NN each heart beat (the same way other volume state is transmitted).
> This will likely be an incompatible change since it requires changing the Datanode protocol.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message