ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez" <afernan...@hortonworks.com>
Subject Re: Review Request 29521: Custom Actions for Namenode to execute tasks by calling a python script
Date Fri, 02 Jan 2015 21:24:37 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29521/
-----------------------------------------------------------

(Updated Jan. 2, 2015, 9:24 p.m.)


Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.


Changes
-------

Addressed comments, waiting for unit test results.


Bugs: AMBARI-8970
    https://issues.apache.org/jira/browse/AMBARI-8970


Repository: ambari


Description
-------

Currently, the upgrade-2.2.xml file, which contains the Upgrade Pack, defines the exact commands
to run for the Namenode prepare and finalize steps.
The downside of having this logic in the xml file is that it makes it difficult to maintain,
and is not as powerful as being able to run the commands by using Python and its functions.
Therefore, it is preferable for the ExecuteTask to be able to invoke a function inside a script.

The plan is for Namenode to have functions for prepare and finalize, and for the upgrade pack
to contain something like,

Masters Group,
'''
    <service name="HDFS">
      <component name="NAMENODE">
        <pre-upgrade>
          <task xsi:type="execute" hosts="master">
            <script>scripts/namenode.py</script>
            <function>prepare_rolling_upgrade</function>
          </task>
        </pre-upgrade>
     </component>
   </service>
'''

While the Cluster Group has,
'''
      <execute-stage service="HDFS" component="NAMENODE" title="Execute HDFS Finalize">
        <task xsi:type="execute" hosts="master">
          <script>scripts/namenode.py</script>
          <function>finalize_rolling_upgrade</function>
        </task>
      </execute-stage>
'''


Diffs (updated)
-----

  ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py d0601a4 
  ambari-agent/src/main/python/ambari_agent/PythonExecutor.py 2ba492f 
  ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 91ddf81 
  ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 51bf45f 
  ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py
7a27322 
  ambari-common/src/main/python/resource_management/libraries/script/script.py 3aa03c9 
  ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
ad86bb1 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
0afc2c00 
  ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java d7bf2de 
  ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteTask.java
a70e08f 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
a2b7c53 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py
PRE-CREATION 
  ambari-server/src/main/resources/custom_actions/scripts/ru_execute_tasks.py a52a7b5 
  ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml dd80134 
  ambari-web/app/messages.js bca3956 

Diff: https://reviews.apache.org/r/29521/diff/


Testing
-------

Deployed a 3-node cluster with HA and performed a rolling upgrade.
There was an error with the Tez client because /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/TEZ/package/scripts/tez_client.py
could not be found, but that is likely unrelated.

Unit tests passed.
{color:green}+1 overall{color}.
  http://issues.apache.org/jira/secure/attachment/12689734/AMBARI-8970.patch

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 2 new or modified
test files.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 core tests{color}.  The patch passed unit tests in ambari-server.


Thanks,

Alejandro Fernandez


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message