Return-Path: 
 <dev-return-53413-apmail-ambari-dev-archive=ambari.apache.org@ambari.apache.org>
X-Original-To: apmail-ambari-dev-archive@www.apache.org
Delivered-To: apmail-ambari-dev-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id D028710290
	for <apmail-ambari-dev-archive@www.apache.org>;
 Fri,  2 Jan 2015 21:43:35 +0000 (UTC)
Received: (qmail 64545 invoked by uid 500); 2 Jan 2015 21:43:36 -0000
Delivered-To: apmail-ambari-dev-archive@ambari.apache.org
Received: (qmail 64503 invoked by uid 500); 2 Jan 2015 21:43:36 -0000
Mailing-List: contact dev-help@ambari.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:dev-help@ambari.apache.org>
List-Unsubscribe: <mailto:dev-unsubscribe@ambari.apache.org>
List-Post: <mailto:dev@ambari.apache.org>
List-Id: <dev.ambari.apache.org>
Reply-To: dev@ambari.apache.org
Delivered-To: mailing list dev@ambari.apache.org
Received: (qmail 64488 invoked by uid 99); 2 Jan 2015 21:43:35 -0000
Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Jan 2015 21:43:35 +0000
Received: from reviews.apache.org (localhost [127.0.0.1])
	by reviews.apache.org (Postfix) with ESMTP id 30A6E1CA924;
	Fri,  2 Jan 2015 21:43:33 +0000 (UTC)
Content-Type: multipart/alternative;
 boundary="===============7246896349852739718=="
MIME-Version: 1.0
Subject: Re: Review Request 29521: Custom Actions for Namenode to execute
 tasks by calling a python script
From: "Nate Cole" <ncole@hortonworks.com>
To: "Nate Cole" <ncole@hortonworks.com>,
 "Dmitro Lisnichenko" <dlysnichenko@hortonworks.com>,
 "Jonathan Hurley" <jhurley@hortonworks.com>
Cc: "Alejandro Fernandez" <afernandez@hortonworks.com>,
 "Ambari" <dev@ambari.apache.org>
Date: Fri, 02 Jan 2015 21:43:33 -0000
Message-ID: <20150102214333.31225.66176@reviews.apache.org>
X-ReviewBoard-URL: https://reviews.apache.org
Auto-Submitted: auto-generated
Sender: "Nate Cole" <noreply@reviews.apache.org>
X-ReviewGroup: Ambari
X-ReviewRequest-URL: https://reviews.apache.org/r/29521/
X-Sender: "Nate Cole" <noreply@reviews.apache.org>
References: <20150102212437.31226.86607@reviews.apache.org>
In-Reply-To: <20150102212437.31226.86607@reviews.apache.org>
Reply-To: "Nate Cole" <ncole@hortonworks.com>
X-ReviewRequest-Repository: ambari

--===============7246896349852739718==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29521/#review66555
-----------------------------------------------------------

Ship it!


Ship It!

- Nate Cole


On Jan. 2, 2015, 4:24 p.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29521/
> -----------------------------------------------------------
> 
> (Updated Jan. 2, 2015, 4:24 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-8970
>     https://issues.apache.org/jira/browse/AMBARI-8970
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Currently, the upgrade-2.2.xml file, which contains the Upgrade Pack, defines the exact commands to run for the Namenode prepare and finalize steps.
> The downside of having this logic in the xml file is that it makes it difficult to maintain, and is not as powerful as being able to run the commands by using Python and its functions. Therefore, it is preferable for the ExecuteTask to be able to invoke a function inside a script.
> 
> The plan is for Namenode to have functions for prepare and finalize, and for the upgrade pack to contain something like,
> 
> Masters Group,
> '''
>     <service name="HDFS">
>       <component name="NAMENODE">
>         <pre-upgrade>
>           <task xsi:type="execute" hosts="master">
>             <script>scripts/namenode.py</script>
>             <function>prepare_rolling_upgrade</function>
>           </task>
>         </pre-upgrade>
>      </component>
>    </service>
> '''
> 
> While the Cluster Group has,
> '''
>       <execute-stage service="HDFS" component="NAMENODE" title="Execute HDFS Finalize">
>         <task xsi:type="execute" hosts="master">
>           <script>scripts/namenode.py</script>
>           <function>finalize_rolling_upgrade</function>
>         </task>
>       </execute-stage>
> '''
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py d0601a4 
>   ambari-agent/src/main/python/ambari_agent/PythonExecutor.py 2ba492f 
>   ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 91ddf81 
>   ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 51bf45f 
>   ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py 7a27322 
>   ambari-common/src/main/python/resource_management/libraries/script/script.py 3aa03c9 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java ad86bb1 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 0afc2c00 
>   ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java d7bf2de 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteTask.java a70e08f 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py a2b7c53 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py PRE-CREATION 
>   ambari-server/src/main/resources/custom_actions/scripts/ru_execute_tasks.py a52a7b5 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml dd80134 
>   ambari-web/app/messages.js bca3956 
> 
> Diff: https://reviews.apache.org/r/29521/diff/
> 
> 
> Testing
> -------
> 
> Deployed a 3-node cluster with HA and performed a rolling upgrade.
> There was an error with the Tez client because /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/TEZ/package/scripts/tez_client.py could not be found, but that is likely unrelated.
> 
> Unit tests passed.
> {color:green}+1 overall{color}.
>   http://issues.apache.org/jira/secure/attachment/12689734/AMBARI-8970.patch
> 
>     {color:green}+1 @author{color}.  The patch does not contain any @author tags.
> 
>     {color:green}+1 tests included{color}.  The patch appears to include 2 new or modified test files.
> 
>     {color:green}+1 javac{color}.  The applied patch does not increase the total number of javac compiler warnings.
> 
>     {color:green}+1 release audit{color}.  The applied patch does not increase the total number of release audit warnings.
> 
>     {color:green}+1 core tests{color}.  The patch passed unit tests in ambari-server.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>


--===============7246896349852739718==--