ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (AMBARI-13755) Express Upgrade: Failed to Downgrade Namenode
Date Fri, 06 Nov 2015 01:52:27 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-13755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992920#comment-14992920
] 

Alejandro Fernandez edited comment on AMBARI-13755 at 11/6/15 1:52 AM:
-----------------------------------------------------------------------

This is happening because the downgrade command to HDFS needs to coincide with the same command
that was used to upgrade it.
HDFS supports two types of upgrades:
1. -rollingUpgrade started  (allows downgrading so that there is no data loss. Downgrade is
either "-rollingUpgrade started" or "-rollingUpgrade downgrade")
2. -upgrade (only allows rollback, so any data that was added since the upgrade will be lost.
Rollback is done with "-rollback" command).

Because Express Upgrade will support only these paths, we will use option 1 for all cases.
HDP 2.1->2.3 (only upgrade)
HDP 2.2->2.3 (both upgrade and downgrade)
HDP 2.2->2.2.* (both upgrade and downgrade)
HDP 2.3->2.3.* (both upgrade and downgrade)



was (Author: afernandez):
This is happening because the downgrade command to HDFS needs to coincide with the same command
that was used to upgrade it.
HDFS supports two types of upgrades:
1. -rollingUpgrade started  (allows downgrading so that there is no data loss. Downgrade is
either "-rollingUpgrade started" or "-rollingUpgrade downgrade")
2. -upgrade (only allows rollback, so any data that was added since the upgrade will be lost.
Rollback is done with "-rollback" command).

Because Express Upgrade will support only these paths, we will use option 1 for all cases.
HDP 2.1->2.3 (only upgrade)
HDP 2.2->2.3 (both upgrade and downgrade)
HDP 2.2->2.2.* (both upgrade and downgrade)
HDP 2.3->2.3.* (both upgrade and downgrade)


> Express Upgrade: Failed to Downgrade Namenode
> ---------------------------------------------
>
>                 Key: AMBARI-13755
>                 URL: https://issues.apache.org/jira/browse/AMBARI-13755
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 2.2.0, 2.1.3
>            Reporter: Alejandro Fernandez
>            Assignee: Alejandro Fernandez
>             Fix For: 2.2.0, 2.1.3
>
>
> Setup:
> # Express Upgrade on HDP 2.2.6.0 -> HDP 2.3.0.0
> # Services: HDFS + ZK
> # Upgrade completes successfully
> # Downgrade after reaching Finalize Upgrade
> # Downgrade blocked at Namenode restart 
> Logs:
> {noformat}
> 2015-10-30 01:27:29,563 WARN  namenode.FSNamesystem (FSNamesystem.java:loadFromDisk(750))
- Encountered exception loading fsimage
> org.apache.hadoop.hdfs.server.common.IncorrectVersionException: Unexpected version of
storage directory /hadoop/hdfs/namenode. Reported: -63. Expecting = -60.
>         at org.apache.hadoop.hdfs.server.common.StorageInfo.setLayoutVersion(StorageInfo.java:178)
>         at org.apache.hadoop.hdfs.server.common.StorageInfo.setFieldsFromProperties(StorageInfo.java:131)
>         at org.apache.hadoop.hdfs.server.namenode.NNStorage.setFieldsFromProperties(NNStorage.java:610)
>         at org.apache.hadoop.hdfs.server.namenode.NNStorage.readProperties(NNStorage.java:639)
>         at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:326)
>         at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:202)
>         at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1032)
>         at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:748)
>         at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:538)
>         at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:597)
>         at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:764)
>         at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:748)
>         at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1441)
>         at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1507)
> 2015-10-30 01:27:29,585 INFO  mortbay.log (Slf4jLog.java:info(67)) - Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@jay-cfg-1.c.pramod-thangali.internal:50070
> 2015-10-30 01:27:29,587 WARN  http.HttpServer2 (HttpServer2.java:isRunning(460)) - HttpServer
Acceptor: isRunning is false. Rechecking.
> 2015-10-30 01:27:29,587 WARN  http.HttpServer2 (HttpServer2.java:isRunning(469)) - HttpServer
Acceptor: isRunning is false
> 2015-10-30 01:27:29,595 INFO  impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(210))
- Stopping NameNode metrics system...
> 2015-10-30 01:27:29,596 INFO  impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(216))
- NameNode metrics system stopped.
> 2015-10-30 01:27:29,596 INFO  impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(605))
- NameNode metrics system shutdown complete.
> 2015-10-30 01:27:29,596 FATAL namenode.NameNode (NameNode.java:main(1512)) - Failed to
start namenode.
> org.apache.hadoop.hdfs.server.common.IncorrectVersionException: Unexpected version of
storage directory /hadoop/hdfs/namenode. Reported: -63. Expecting = -60.
>         at org.apache.hadoop.hdfs.server.common.StorageInfo.setLayoutVersion(StorageInfo.java:178)
>         at org.apache.hadoop.hdfs.server.common.StorageInfo.setFieldsFromProperties(StorageInfo.java:131)
>         at org.apache.hadoop.hdfs.server.namenode.NNStorage.setFieldsFromProperties(NNStorage.java:610)
>         at org.apache.hadoop.hdfs.server.namenode.NNStorage.readProperties(NNStorage.java:639)
>         at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:326)
>         at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:202)
>         at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1032)
>         at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:748)
>         at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:538)
>         at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:597)
>         at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:764)
>         at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:748)
>         at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1441)
>         at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1507)
> 2015-10-30 01:27:29,600 INFO  util.ExitUtil (ExitUtil.java:terminate(124)) - Exiting
with status 1
> 2015-10-30 01:27:29,605 INFO  namenode.NameNode (StringUtils.java:run(659)) - SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down NameNode at jay-cfg-1.c.pramod-thangali.internal/10.240.2.108
> ************************************************************/
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message