hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2971) some improvements to the manual NN metadata recovery tools
Date Thu, 23 Feb 2012 19:32:49 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214968#comment-13214968

Colin Patrick McCabe commented on HDFS-2971:

Hi Suresh,

After thinking about it, I believe that I agree with you.  We should check that generation
stamps are non-decreasing during normal loading, rather than only doing it in OEV.  Todd filed
HDFS-2872 to address this, and I'll try to implement it.  I will close this JIRA issue.

I'm curious if you have any thoughts or suggestions about https://issues.apache.org/jira/browse/HDFS-3004.
 Basically, the idea is to have something similar to a traditional fsck for the HDFS EditLog
and Image.

We obviously would prefer never to get into trouble with these files in the first place, but
hardware being hardware, sometimes we do.  It would be preferrable to have a semi-automated
way to fix inconsistencies and corruption in these files rather than leaving it to the programmer
using a hex editor, which is the current situation.  I'm curious if you have any suggestions
for this.
> some improvements to the manual NN metadata recovery tools
> ----------------------------------------------------------
>                 Key: HDFS-2971
>                 URL: https://issues.apache.org/jira/browse/HDFS-2971
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: libhdfs
>    Affects Versions: 1.1.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: 2012-02-17_0001-OEV-enhancements-pt.-2.patch, HDFS-2971__print_highest_generation_stamp.txt
> Some improvements to the manual NN metadata recovery tools.
> Specifically, we want the Offline Edit Viewer (oev) tool to prints out the highest generation
stamp that was encountered when processing the edit log.
> We also want OEV to look for large gaps in the generation stamp, as these can indicate
corruption.  The minimum gap to look for should be configurable with -G or --genStampGap.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message