Return-Path: 
 <dev-return-53470-apmail-ambari-dev-archive=ambari.apache.org@ambari.apache.org>
X-Original-To: apmail-ambari-dev-archive@www.apache.org
Delivered-To: apmail-ambari-dev-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id AC1C0C30F
	for <apmail-ambari-dev-archive@www.apache.org>;
 Mon,  5 Jan 2015 11:57:20 +0000 (UTC)
Received: (qmail 93806 invoked by uid 500); 5 Jan 2015 11:57:21 -0000
Delivered-To: apmail-ambari-dev-archive@ambari.apache.org
Received: (qmail 93765 invoked by uid 500); 5 Jan 2015 11:57:21 -0000
Mailing-List: contact dev-help@ambari.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:dev-help@ambari.apache.org>
List-Unsubscribe: <mailto:dev-unsubscribe@ambari.apache.org>
List-Post: <mailto:dev@ambari.apache.org>
List-Id: <dev.ambari.apache.org>
Reply-To: dev@ambari.apache.org
Delivered-To: mailing list dev@ambari.apache.org
Received: (qmail 93750 invoked by uid 99); 5 Jan 2015 11:57:20 -0000
Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2015 11:57:20 +0000
Received: from reviews.apache.org (localhost [127.0.0.1])
	by reviews.apache.org (Postfix) with ESMTP id 2FC801CABF7;
	Mon,  5 Jan 2015 11:57:18 +0000 (UTC)
Content-Type: multipart/alternative;
 boundary="===============0196449562974853461=="
MIME-Version: 1.0
Subject: Re: Review Request 29521: Custom Actions for Namenode to execute
 tasks by calling a python script
From: "Dmitro Lisnichenko" <dlysnichenko@hortonworks.com>
To: "Nate Cole" <ncole@hortonworks.com>,
 "Dmitro Lisnichenko" <dlysnichenko@hortonworks.com>,
 "Jonathan Hurley" <jhurley@hortonworks.com>
Cc: "Alejandro Fernandez" <afernandez@hortonworks.com>,
 "Ambari" <dev@ambari.apache.org>
Date: Mon, 05 Jan 2015 11:57:18 -0000
Message-ID: <20150105115718.26560.71430@reviews.apache.org>
X-ReviewBoard-URL: https://reviews.apache.org
Auto-Submitted: auto-generated
Sender: "Dmitro Lisnichenko" <noreply@reviews.apache.org>
X-ReviewGroup: Ambari
X-ReviewRequest-URL: https://reviews.apache.org/r/29521/
X-Sender: "Dmitro Lisnichenko" <noreply@reviews.apache.org>
References: <20150103002112.31226.95972@reviews.apache.org>
In-Reply-To: <20150103002112.31226.95972@reviews.apache.org>
Reply-To: "Dmitro Lisnichenko" <dlysnichenko@hortonworks.com>
X-ReviewRequest-Repository: ambari

--===============0196449562974853461==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29521/#review66621
-----------------------------------------------------------

Ship it!


Ship It!

- Dmitro Lisnichenko


On Jan. 3, 2015, 12:21 a.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29521/
> -----------------------------------------------------------
> 
> (Updated Jan. 3, 2015, 12:21 a.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-8970
>     https://issues.apache.org/jira/browse/AMBARI-8970
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Currently, the upgrade-2.2.xml file, which contains the Upgrade Pack, defines the exact commands to run for the Namenode prepare and finalize steps.
> The downside of having this logic in the xml file is that it makes it difficult to maintain, and is not as powerful as being able to run the commands by using Python and its functions. Therefore, it is preferable for the ExecuteTask to be able to invoke a function inside a script.
> 
> The plan is for Namenode to have functions for prepare and finalize, and for the upgrade pack to contain something like,
> 
> Masters Group,
> '''
>     <service name="HDFS">
>       <component name="NAMENODE">
>         <pre-upgrade>
>           <task xsi:type="execute" hosts="master">
>             <script>scripts/namenode.py</script>
>             <function>prepare_rolling_upgrade</function>
>           </task>
>         </pre-upgrade>
>      </component>
>    </service>
> '''
> 
> While the Cluster Group has,
> '''
>       <execute-stage service="HDFS" component="NAMENODE" title="Execute HDFS Finalize">
>         <task xsi:type="execute" hosts="master">
>           <script>scripts/namenode.py</script>
>           <function>finalize_rolling_upgrade</function>
>         </task>
>       </execute-stage>
> '''
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py d0601a4 
>   ambari-agent/src/main/python/ambari_agent/PythonExecutor.py 2ba492f 
>   ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 91ddf81 
>   ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 51bf45f 
>   ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py 7a27322 
>   ambari-common/src/main/python/resource_management/libraries/script/script.py 3aa03c9 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java ad86bb1 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 0afc2c00 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java b64e9ce 
>   ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java d7bf2de 
>   ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 2a49e24 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteTask.java a70e08f 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py a2b7c53 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py PRE-CREATION 
>   ambari-server/src/main/resources/custom_actions/scripts/ru_execute_tasks.py a52a7b5 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml dd80134 
>   ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java 5175b87 
>   ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProviderTest.java 6e20048 
>   ambari-web/app/messages.js bca3956 
> 
> Diff: https://reviews.apache.org/r/29521/diff/
> 
> 
> Testing
> -------
> 
> Deployed a 3-node cluster with HA and performed a rolling upgrade.
> There was an error with the Tez client because /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/TEZ/package/scripts/tez_client.py could not be found, but that is likely unrelated.
> 
> Unit tests passed.
> {color:green}+1 overall{color}.
>   http://issues.apache.org/jira/secure/attachment/12689734/AMBARI-8970.patch
> 
>     {color:green}+1 @author{color}.  The patch does not contain any @author tags.
> 
>     {color:green}+1 tests included{color}.  The patch appears to include 2 new or modified test files.
> 
>     {color:green}+1 javac{color}.  The applied patch does not increase the total number of javac compiler warnings.
> 
>     {color:green}+1 release audit{color}.  The applied patch does not increase the total number of release audit warnings.
> 
>     {color:green}+1 core tests{color}.  The patch passed unit tests in ambari-server.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>


--===============0196449562974853461==--