hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6597) Add a new option to NN upgrade to terminate the process after upgrade on NN is completed
Date Tue, 15 Jul 2014 21:37:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062687#comment-14062687

Chris Nauroth commented on HDFS-6597:

Hi, [~dvunjak].  This mostly looks good.  I have 2 comments:
# I built the distro after adding a bogus new feature to bump the version to -58 in {{NameNodeLayoutVersion}}.
 I tried running -upgradeOnly, but it didn't actually upgrade the metadata files.  It looks
like you'll need another change in {{FSImage#recoverTransitionRead}}.  There is a switch statement
that looks for the {{UPGRADE}} option, but not the new {{UPGRADEONLY}} option.
# The new test suite is a copy of the existing {{TestStartupOptionUpgrade}} with the option
changed to {{UPGRADEONLY}}.  Instead of cloning the code, this looks like a good opportunity
for a JUnit {{Parameterized}} test.  See {{TestNameNodeHttpServer}} for an existing example
of a {{Parameterized}} test.  I think you can make a fairly small change in the existing {{TestStartupOptionUpgrade}}
so that it's parameterized to run on both options: {{UPGRADE}} and {{UPGRADEONLY}}.

> Add a new option to NN upgrade to terminate the process after upgrade on NN is completed
> ----------------------------------------------------------------------------------------
>                 Key: HDFS-6597
>                 URL: https://issues.apache.org/jira/browse/HDFS-6597
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>            Reporter: Danilo Vunjak
>            Assignee: Danilo Vunjak
>         Attachments: HDFS-6597.04.patch, JIRA-HDFS-30.patch, JIRA-HDFS-6597.02.patch,
JIRA-HDFS-6597.03.patch, JIRA-HDFS-6597.patch
> Currently when namenode is started for upgrade (hadoop namenode -upgrade command), after
finishing upgrade of metadata, namenode starts working normally and wait for datanodes to
upgrade itself and connect to to NN. We need to have option for upgrading only NN metadata,
so after upgrade is finished on NN, process should terminate.
> I have tested it by changing in file: hdfs.server.namenode.NameNode.java, method: public
static NameNode createNameNode(String argv[], Configuration conf):
>  in switch added
>  case UPGRADE:
> case UPGRADE:
>       {
>     	  DefaultMetricsSystem.initialize("NameNode");
>           NameNode nameNode = new NameNode(conf);
>           if (startOpt.getForceUpgrade()) {
>         	  terminate(0);
>         	  return null;
>           }
>           return nameNode;
>       }
> This did upgrade of metadata, closed process after finished, and later when all services
were started, upgrade of datanodes finished sucessfully and system run .
> What I'm suggesting right now is to add new startup parameter "-force", so namenode can
be started like this "hadoop namenode -upgrade -force", so we can indicate that we want to
terminate process after upgrade metadata on NN is finished. Old functionality should be preserved,
so users can run "hadoop namenode -upgrade" on same way and with same behaviour as it was
>  Thanks,
>  Danilo

This message was sent by Atlassian JIRA

View raw message