ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hurley" <jhur...@hortonworks.com>
Subject Re: Review Request 39111: [PREVIEW] Stop-and-Start Upgrade: Handle Core Services
Date Thu, 08 Oct 2015 16:08:11 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39111/#review101925
-----------------------------------------------------------

Ship it!



ambari-common/src/main/python/resource_management/libraries/script/script.py (lines 439 -
442)
<https://reviews.apache.org/r/39111/#comment159421>

    Why do we we need to convert from foo_upgrade to foo here; we should be using consistent
naming conventions across Java and python. I'd say just use "rolling" or "nonrolling" for
the restart_type.



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
(line 594)
<https://reviews.apache.org/r/39111/#comment159422>

    Diamond operator is OK here - no need to change it.



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
(line 936)
<https://reviews.apache.org/r/39111/#comment159423>

    Can you specify what extra parameters are and provide an example; to another engineer
looking at this, it's hard to understand.



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
(lines 936 - 945)
<https://reviews.apache.org/r/39111/#comment159424>

    Can this code be abstracted into a method? It's cut/pasted a few times.



ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java (line 139)
<https://reviews.apache.org/r/39111/#comment159425>

    Is "this" needed here?



ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java (line 267)
<https://reviews.apache.org/r/39111/#comment159426>

    Diamond operator is OK.



ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java (line 380)
<https://reviews.apache.org/r/39111/#comment159427>

    Diamond operator is OK.



ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
(lines 71 - 79)
<https://reviews.apache.org/r/39111/#comment159428>

    Any reason you didn't make this generic; something like `get_binary_path` in case it needs
to be re-used for other components?



ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
(line 156)
<https://reviews.apache.org/r/39111/#comment159429>

    "Express Upgrade"



ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py
(lines 183 - 189)
<https://reviews.apache.org/r/39111/#comment159430>

    Documentation


- Jonathan Hurley


On Oct. 7, 2015, 8 p.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39111/
> -----------------------------------------------------------
> 
> (Updated Oct. 7, 2015, 8 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, and
Nate Cole.
> 
> 
> Bugs: AMBARI-12701
>     https://issues.apache.org/jira/browse/AMBARI-12701
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Stop-and-Start Upgrade of the Stack to handle core services: HDFS, YARN, MR, ZK
> 
> * Renamed the functions pre_rolling_upgrade and post_rolling_upgrade to pre_upgrade_restart
and post_upgrade_restart that accept the upgrade_type (rolling|nonrolling) as a param.
> 
> * When NameNode HA is present, the command used to start NameNode is different between
the active and the standby, so changed MasterHostResolver.java to pick one of the NameNodes
to become the active and the other the standby, and pass it in a parameter called "desired_namenode_role"
> 
> * Wrote a function to take a backup of HDFS
> 
> * Changed the "hdfs" command to actually search which one to use depending on the stack
and version to account for Express Upgrade from HDP 2.1 -> 2.3
> 
> * Tested Express Upgrade with and without NameNode HA
> 
> 
> Diffs
> -----
> 
>   ambari-common/src/main/python/resource_management/libraries/script/script.py 26fd158

>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
a2f642c 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
ef75d38 
>   ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java ed9b3a7

>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java
648e7a4 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ColocatedGrouping.java
a8e9c43 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java
1164335 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
4116101 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
68fec03 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapper.java
92df3b5 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapperBuilder.java
6c0f3c7 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/TaskWrapper.java
f7cc930 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/TaskWrapperBuilder.java
a5813e3 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py
fa68435 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
2e5ac19 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py
dd0dca4 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
b11d7ea 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py
46c7272 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py
e2ebbcb 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
07be496 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py
e8c142c 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py
fb39878 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/nfsgateway.py
be6f0d5 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
5f6b8fe 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/setup_ranger_hdfs.py
f5df86f 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/snamenode.py
108ef01 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py
7bc0b6a 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
14de094 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py
f3b3d11 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py
16e34d4 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py
5263d9f 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py
fd25651 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
f3914f3 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py
e67f1ce 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py
e58ea3c 
>   ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper.py
973fa0f 
>   ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_client.py
02c1006 
>   ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py
b7fb578 
>   ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_service.py
685eb6d 
> 
> Diff: https://reviews.apache.org/r/39111/diff/
> 
> 
> Testing
> -------
> 
> * Tested Express Upgrade with and without NameNode HA
> 
> There are still a handful of bugs.
> * YARN history server start fails because NameNode is still in safemode and waiting for
block reports.
> * YARN Service Check fails due to some bad properties in the config pack from 2.1->2.3
> * May need to remove HDP 2.1
> * Taking HDFS backup and report to /tmp folder
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message