hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Prakash (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2848) hdfs corruption appended to blocks is not detected by fs commands or fsck
Date Fri, 03 Feb 2012 15:57:54 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199795#comment-13199795

Ravi Prakash commented on HDFS-2848:

Thanks Kihwal, Nathan, Daryn and Bobby for all your help. :)

We've determined that this bug will be closed as Won't Fix because:
1. The periodic DirectoryScanner (used for block reports every 6 hours) DOES detect the corruption.

2. During the time between the DirectoryScanner runs, the DN continues to serve the correct,
uncorrupted data. 
3. If the DN is restarted between the DirectoryScanner runs, corruption is again detected.
4. Appends work fine and overwrite the corrupt data. 

So bad data is never served. And corruption is detected frequently if not instantaneously.

If anyone has any objections, please speak now. Or forever hold your peace :-D
> hdfs corruption appended to blocks is not detected by fs commands or fsck
> -------------------------------------------------------------------------
>                 Key: HDFS-2848
>                 URL: https://issues.apache.org/jira/browse/HDFS-2848
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 0.23.0
>            Reporter: Ravi Prakash
>            Assignee: Ravi Prakash
> Courtesy Pat White [~patwhitey2007]
> {quote}
> Appears that there is a regression in corrupt block detection by both fsck and fs cmds
like 'cat'. Testcases for
> pre-block and block-overwrite corruption of all replicas is correctly reporting errors
however post-block corruption is
> not, fsck on the filesystem reports it's Healthy and 'cat' returns without error. Looking
at the DN blocks themselves,
> they clearly contain the injected corruption pattern.
> {quote}

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message