ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez" <afernan...@hortonworks.com>
Subject Re: Review Request 39111: AMBARI-12701. Stop-and-Start Upgrade: Handle Core Services
Date Thu, 22 Oct 2015 00:51:00 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39111/
-----------------------------------------------------------

(Updated Oct. 22, 2015, 12:50 a.m.)


Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, and Nate
Cole.


Summary (updated)
-----------------

AMBARI-12701. Stop-and-Start Upgrade: Handle Core Services


Bugs: AMBARI-12701
    https://issues.apache.org/jira/browse/AMBARI-12701


Repository: ambari


Description
-------

Stop-and-Start Upgrade of the Stack to handle core services: HDFS, YARN, MR, ZK

* Renamed the functions pre_rolling_upgrade and post_rolling_upgrade to pre_upgrade_restart
and post_upgrade_restart that accept the upgrade_type (rolling|nonrolling) as a param.

* When NameNode HA is present, the command used to start NameNode is different between the
active and the standby, so changed MasterHostResolver.java to pick one of the NameNodes to
become the active and the other the standby, and pass it in a parameter called "desired_namenode_role"

* Wrote a function to take a backup of HDFS

* Changed the "hdfs" command to actually search which one to use depending on the stack and
version to account for Express Upgrade from HDP 2.1 -> 2.3

* Tested Express Upgrade with and without NameNode HA


Diffs
-----

  ambari-common/src/main/python/resource_management/libraries/script/script.py efd34e3 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PreUpgradeCheckResourceProvider.java
9c5d87a 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
ecf22c2 
  ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 55fb12b

  ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java b81d3fd 
  ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java
ba44408 
  ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ColocatedGrouping.java
2aef43c 
  ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java 8f23803

  ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
af63656 
  ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapper.java
92df3b5 
  ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapperBuilder.java
47a28d7 
  ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/TaskWrapper.java
f7cc930 
  ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/TaskWrapperBuilder.java
a5813e3 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py
fa68435 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
2e5ac19 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py
dd0dca4 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
b11d7ea 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py
46c7272 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py
e2ebbcb 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
23e775a 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py
e8c142c 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py
fb39878 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/nfsgateway.py
be6f0d5 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
429ffa9 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/setup_ranger_hdfs.py
f5df86f 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/snamenode.py
108ef01 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py
d1e764a 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
14de094 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py
f3b3d11 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py
229dcd8 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py
5263d9f 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py
fd25651 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
c39e4c4 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py
e67f1ce 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py
e58ea3c 
  ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper.py
973fa0f 
  ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_client.py
02c1006 
  ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py
b7fb578 
  ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_service.py
b983983 
  ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 3778655

  ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/config-upgrade.xml fbd21a4 
  ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml 2f6840f

  ambari-server/src/test/java/org/apache/ambari/server/state/stack/upgrade/StageWrapperBuilderTest.java
bac00d4 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py e624eea 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_hdfs_client.py 4948d01 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py becc82b 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 68d7d62 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_nfsgateway.py 62458c1 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 871d313 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_client.py 0e52264 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 042797b 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py fb7d847 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_yarn_client.py 21fbb9d 
  ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_client.py 79fd74b 
  ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_server.py afc4bc7 
  ambari-server/src/test/python/stacks/2.1/YARN/test_apptimelineserver.py cb7b06b 

Diff: https://reviews.apache.org/r/39111/diff/


Testing
-------

* Tested Express Upgrade with and without NameNode HA

There are still a handful of bugs.
* YARN history server start fails because NameNode is still in safemode and waiting for block
reports.
* YARN Service Check fails due to some bad properties in the config pack from 2.1->2.3
* May need to remove HDP 2.1
* Taking HDFS backup and report to /tmp folder


Thanks,

Alejandro Fernandez


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message