hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1371) One bad node can incorrectly flag many files as corrupt
Date Sun, 05 Sep 2010 00:42:35 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906315#action_12906315
] 

dhruba borthakur commented on HDFS-1371:
----------------------------------------

> e. I think a better solution could be client notify the datanode of the possible corruption
and let DN to doublecheck and report to NN.

any drawbacks of this approach? Keeps namenode complexity down, prevents DOS, etc.

> One bad node can incorrectly flag many files as corrupt
> -------------------------------------------------------
>
>                 Key: HDFS-1371
>                 URL: https://issues.apache.org/jira/browse/HDFS-1371
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs client, name-node
>    Affects Versions: 0.20.1
>         Environment: yahoo internal version 
> [knoguchi@gwgd4003 ~]$ hadoop version
> Hadoop 0.20.104.3.1007030707
>            Reporter: Koji Noguchi
>
> On our cluster, 12 files were reported as corrupt by fsck even though the replicas on
the datanodes were healthy.
> Turns out that all the replicas (12 files x 3 replicas per file) were reported corrupt
from one node.
> Surprisingly, these files were still readable/accessible from dfsclient (-get/-cat) without
any problems.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message