hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "HariSree (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-702) DFS Upgrade Proposal
Date Mon, 30 May 2011 12:16:47 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041105#comment-13041105

HariSree commented on HADOOP-702:

What is the expectation in the following scenario : 

Namenode has 3 name dirs configured ..
1) Namenode upgrade starts - Upgrade fails after 1st directory is upgraded  (2nd and 3rd dir
is left unchanged ..)
2) Namenode starts 
3) Namenode shutdown and rollbacked

Are the new changes done meant to be visible ? With current implementation , after a rollback
new changes are visible .. Is this expected ??

As per my observation , since Namenode is saving the latest image dir(the upgraded 1st dir
since checkpointtime is incremented during upgrade for this dir) will be loaded and saved
to all dirs during loadfsimage  .. 

But if a ROLLBACK is done , the 1st dir will be rolled back (the older copy becomes current
and its checkpointtime is now LESS than other dirs ..) and others left behind since they dont
contain previous .. Now during loadfsimage , the 2nd dir will be selected since it has the
highest checkpoint time and saved to all dirs (including 1st ) .. Now due to this , the new
changes b/w UPGRADE and ROLLBACK present in 2nd dir gets reflected even after ROLLBACK ..

Is this expected ? I ask this since  : 

1) this is not the case with a SUCCESSFULL Upgrade/Rollback (New changes lost after rollback)..
2) FSStateTransition7.htm says { r0. if previous directory does not exist then fail rollback
} , but it continues in the current implementation .. 

> DFS Upgrade Proposal
> --------------------
>                 Key: HADOOP-702
>                 URL: https://issues.apache.org/jira/browse/HADOOP-702
>             Project: Hadoop Common
>          Issue Type: New Feature
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: 0.13.0
>         Attachments: DFSUpgradeProposal3.html, FSStateTransition6.htm, FSStateTransition7.htm,
FSStateTransitionApr03.patch, Manual-TestCases-HdfsConversion.txt, TestPlan-HdfsUpgrade.html,
TestPlan-HdfsUpgrade.html, TestPlan-HdfsUpgrade.html
> Currently the DFS cluster upgrade procedure is manual.
> http://wiki.apache.org/lucene-hadoop/Hadoop_Upgrade
> It is rather complicated and does not guarantee data recoverability in case of software
errors or administrator mistakes.
> This is a description of utilities that make the upgrade process almost automatic and
minimize chance of loosing or corrupting data.
> Please see the attached html file for details.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message