hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jakob Homan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-215) Offline Namenode fsImage verification
Date Thu, 02 Jul 2009 21:35:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12726693#action_12726693
] 

Jakob Homan commented on HDFS-215:
----------------------------------

Now that the offline image viewer (oiv) is up and running, it can certainly do this.  Running
any of the image processors against an image and completing successfully indicates the image
is good.  The oiv will bail if it runs into anything unexpected.  It's pretty fast with the
current processors, but if a faster one were needed, a processor could be written that didn't
generate any actual output, just processed the file checking that it got all the elements
in the order expected and report its success at the end.


> Offline Namenode fsImage verification
> -------------------------------------
>
>                 Key: HDFS-215
>                 URL: https://issues.apache.org/jira/browse/HDFS-215
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Brian Bockelman
>
> Currently, there is no way to verify that a copy of the fsImage is not corrupt.  I propose
that we should have an offline tool that loads the fsImage into memory to see if it is usable.
 This will allow us to automate backup testing to some extent.
> One can start a namenode process on the fsImage to see if it can be loaded, but this
is not easy to automate.
> To use HDFS in production, it is greatly desired to have both checkpoints - and have
some idea that the checkpoints are valid!  No one wants to see the day where they reload from
backup only to find that the fsImage in the backup wasn't usable.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message