Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Fri, 6 Apr 2012 01:26:23 +0000 (UTC)
From: "Colin Patrick McCabe (Commented) (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: 
 <1645880450.21066.1333675583994.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <837427371.11003.1330025088720.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HDFS-3004) Implement Recovery Mode
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-3004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247942#comment-13247942 ] 

Colin Patrick McCabe commented on HDFS-3004:
--------------------------------------------

Hi Nicholas,

If the -noPrompt option prompts, it will be impossible for the unit tests to use it to test recovery.  Basically, we need a way of running recovery without human intervention.

We don't prompt before wiping the entire fsimage when the user specifies -format.  I don't see why -recover should operate any differently, especially when the user has to specify both -recover and -noPrompt in order to have the potential of data loss without a prompt.

We don't want to get in the habit of presenting the user with lots of nag messages that he'll just blindly say "yes" to out of annoyance.  The best practice is to back up the edit log and image directories before modifying them, just the same as now.

As far as flag naming: yeah, --force might work.  I'll think about it and give others a chance to make suggestions as well.

thanks,
C.
                
> Implement Recovery Mode
> -----------------------
>
>                 Key: HDFS-3004
>                 URL: https://issues.apache.org/jira/browse/HDFS-3004
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: tools
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: HDFS-3004.010.patch, HDFS-3004.011.patch, HDFS-3004.012.patch, HDFS-3004.013.patch, HDFS-3004.015.patch, HDFS-3004.016.patch, HDFS-3004.017.patch, HDFS-3004.018.patch, HDFS-3004.019.patch, HDFS-3004.020.patch, HDFS-3004.022.patch, HDFS-3004.023.patch, HDFS-3004.024.patch, HDFS-3004.026.patch, HDFS-3004.027.patch, HDFS-3004.029.patch, HDFS-3004.030.patch, HDFS-3004.031.patch, HDFS-3004.032.patch, HDFS-3004.033.patch, HDFS-3004.034.patch, HDFS-3004.035.patch, HDFS-3004.036.patch, HDFS-3004.037.patch, HDFS-3004.038.patch, HDFS-3004.039.patch, HDFS-3004.040.patch, HDFS-3004__namenode_recovery_tool.txt
>
>
> When the NameNode metadata is corrupt for some reason, we want to be able to fix it.  Obviously, we would prefer never to get in this case.  In a perfect world, we never would.  However, bad data on disk can happen from time to time, because of hardware errors or misconfigurations.  In the past we have had to correct it manually, which is time-consuming and which can result in downtime.
> Recovery mode is initialized by the system administrator.  When the NameNode starts up in Recovery Mode, it will try to load the FSImage file, apply all the edits from the edits log, and then write out a new image.  Then it will shut down.
> Unlike in the normal startup process, the recovery mode startup process will be interactive.  When the NameNode finds something that is inconsistent, it will prompt the operator as to what it should do.   The operator can also choose to take the first option for all prompts by starting up with the '-f' flag, or typing 'a' at one of the prompts.
> I have reused as much code as possible from the NameNode in this tool.  Hopefully, the effort that was spent developing this will also make the NameNode editLog and image processing even more robust than it already is.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira