Mailing-List: contact dev-help@ambari.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ambari.apache.org
Content-Type: multipart/alternative;
 boundary="===============1138876971353296413=="
MIME-Version: 1.0
Subject: Re: Review Request 36028: RU - Misc issues: ZKFC not upgraded on
 Standby NN; Flume kill needs signal;
 Package Installation fails when host has no Stack components
From: "Alejandro Fernandez" <afernandez@hortonworks.com>
To: "Nate Cole" <ncole@hortonworks.com>,
 "Dmitro Lisnichenko" <dlysnichenko@hortonworks.com>,
 "Jonathan Hurley" <jhurley@hortonworks.com>
Cc: "Alejandro Fernandez" <afernandez@hortonworks.com>,
 "Ambari" <dev@ambari.apache.org>
Date: Tue, 30 Jun 2015 00:20:01 -0000
Message-ID: <20150630002001.13307.99057@reviews.apache.org>
Auto-Submitted: auto-generated
Sender: "Alejandro Fernandez" <noreply@reviews.apache.org>
References: <20150630001358.13308.97700@reviews.apache.org>
In-Reply-To: <20150630001358.13308.97700@reviews.apache.org>
Reply-To: "Alejandro Fernandez" <afernandez@hortonworks.com>

--===============1138876971353296413==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36028/#review89831
-----------------------------------------------------------


ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume.py (line 207)
<https://reviews.apache.org/r/36028/#comment142644>

    Fix the flume issue by supplying the signal to the kill command.


ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py (line 84)
<https://reviews.apache.org/r/36028/#comment142645>

    Now, we try to kill ZKFC in both standby and active. This will allow starting ZKFC in both cases using the newer version.


ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py (line 91)
<https://reviews.apache.org/r/36028/#comment142646>

    If killed the active, need to wait for the failover to happen before continuing.


ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py (line 126)
<https://reviews.apache.org/r/36028/#comment142647>

    -15 is safer than -9


ambari-server/src/main/resources/custom_actions/scripts/install_packages.py (line 91)
<https://reviews.apache.org/r/36028/#comment142648>

    This should always be present.


ambari-server/src/main/resources/custom_actions/scripts/install_packages.py (line 242)
<https://reviews.apache.org/r/36028/#comment142649>

    Handle the case of a host that only contains AMS.


- Alejandro Fernandez


On June 30, 2015, 12:13 a.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36028/
> -----------------------------------------------------------
> 
> (Updated June 30, 2015, 12:13 a.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-12205
>     https://issues.apache.org/jira/browse/AMBARI-12205
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Fix 3 issues related to Rolling Upgrades.
> 
> 1. RU orchestration first upgrades the standby NameNode, then the active. However,  HDFS utils.py only kills ZKFC on the active NameNode; instead, this should be on both active and standby.
> 2. When Ambari installs the packages of an HDP repo before RU, if the host only has AMS, then "hdp-select versions" will not report any values. Technically, this is ok for this host, so Ambari should ignore this. However, if the host does not report any delta of versions and was not able to read a version from /var/lib/ambari-agent/data/repo_version_history.json and does contain the stack's root folder (e.g., /usr/hdp), then it should result in an error.
> 3. In RU, the flume stop command issues a kill, but does not supply a signal, such as -15
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume.py 226cf8c 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 87c38ae 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2360d7a 
>   ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 8d1c07f 
> 
> Diff: https://reviews.apache.org/r/36028/diff/
> 
> 
> Testing
> -------
> 
> ----------------------------------------------------------------------
> Total run:766
> Total errors:0
> Total failures:0
> OK
> 
> Verified that the patch works.
> 
> 1. ZKFC is upgraded on both NameNode hosts:
> 
> Before RU:
> `
> ps -ef | grep zkfc
> hdfs      2151     1  2 23:23 ?        00:00:02 /usr/jdk64/jdk1.8.0_40/bin/java -Dproc_zkfc -Xmx1024m -Dhdp.version=2.2.7.0-2878
> `
> 
> After RU:
> `
> ps -ef | grep zkfc
> hdfs      4069     1  3 23:28 ?        00:00:02 /usr/jdk64/jdk1.8.0_40/bin/java -Dproc_zkfc -Xmx1024m -Dhdp.version=2.3.0.0-2458
> `
> 
> 2. Logs show that host with only AMS was successful,
> `
> 2015-06-29 22:43:18,046 - Attempting to determine actual version with build number.
> 2015-06-29 22:43:18,047 - Old versions: []
> 2015-06-29 22:43:18,047 - call['hdp-select versions'] {}
> 2015-06-29 22:43:18,067 - call returned (1, 'Traceback (most recent call last):\n  File "/usr/bin/hdp-select", line 377, in <module>\n    printVersions()\n  File "/usr/bin/hdp-select", line 232, in printVersions\n    for f in os.listdir(root):\nOSError: [Errno 2] No such file or directory: \'/usr/hdp\'')
> 2015-06-29 22:43:18,067 - New versions: []
> 2015-06-29 22:43:18,067 - Deltas: set([])
> 2015-06-29 22:43:18,067 - Cannot determine a new actual version installed by using the delta method. This is expected during additional attempts to install packages since "hdp-select versions" will not report a new value.
> 2015-06-29 22:43:18,067 - This stack's root directory (/usr/hdp) is not present on this host, so this host does not contain any versionable components.Therefore, ignore this host and allow other hosts to report the correct repository version.
> `
> 
> 
> 3. Killed Flume successfully,
> `
> 2015-06-29 23:47:28,428 - Execute['('kill', '-15', '11987')'] {'sudo': True}
> `
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>


--===============1138876971353296413==--