ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmytro Sen" <d...@hortonworks.com>
Subject Re: Review Request 16965: Allow log4j properties to be applied via the API in Ambari for hadoop/oozie/hbase/hive/zookeeper/pig
Date Thu, 23 Jan 2014 01:28:59 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16965/
-----------------------------------------------------------

(Updated Jan. 23, 2014, 1:28 a.m.)


Review request for Ambari, Andrew Onischuk, Oleksandr Diachenko, and Sumit Mohanty.


Bugs: AMBARI-3537
    https://issues.apache.org/jira/browse/AMBARI-3537


Repository: ambari


Description
-------


New cluster deployment scenario:
- During cluster deployment UI can read default values from "....\resources\stacks\HDP\2.0.8\services\HDFS\configuration\hdfs-log4j.xml",
but user is able to modify them before deployment.
- UI makes calls like
     curl -i -X PUT -d '{"Clusters":{"desired_configs":{"type":"hdfs-log4j","tag":"version1","properties":{"hadoop.root.logger":"INFO,console"}}}}'
http://localhost:8080/api/v1/clusters/c1
- python scripts at the agent side store hdfs-log4j configs as log4j.properties in /etc/hadoop/conf/
- log4j.properties management is made by Ambari UI or API

Cluster upgrade scenario(not implemented yet):
- before running "ambari-server upgrade" command, user modifies *-log4j.xml files at the upgrade
stack directory (like stacks\HDP\2.0.8\services\HDFS\configuration)
- run ambari-server upgrade


After applying the patch, ambari server is able to manage log4j properties at the files below:
/etc/hadoop/conf/log4j.properties
/etc/hbase/conf/log4j.properties
/etc/hive/conf/hive-log4j.properties.template
/etc/hive/conf/hive-log4j.properties.template
/etc/oozie/conf/oozie-log4j.properties
/etc/pig/conf/log4j.properties
/etc/zookeeper/conf/log4j.properties


Diffs
-----

  ambari-agent/src/main/python/resource_management/libraries/providers/__init__.py c5e7bae

  ambari-agent/src/main/python/resource_management/libraries/providers/properties_file.py
PRE-CREATION 
  ambari-agent/src/main/python/resource_management/libraries/resources/__init__.py bad8470

  ambari-agent/src/main/python/resource_management/libraries/resources/properties_file.py
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/hooks/before-START/scripts/params.py aabb406

  ambari-server/src/main/resources/stacks/HDP/1.3.3/hooks/before-START/scripts/shared_initialization.py
7b406e1 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/hooks/before-START/templates/log4j.properties.j2
577ad04 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HBASE/configuration/hbase-log4j.xml
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HBASE/package/scripts/hbase.py
bd33463 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HBASE/package/scripts/params.py
95880cb 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HDFS/configuration/hdfs-log4j.xml
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HIVE/configuration/hive-exec-log4j.xml
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HIVE/configuration/hive-log4j.xml
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HIVE/package/scripts/hive.py
b37ebb2 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HIVE/package/scripts/params.py
0cf89be 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/MAPREDUCE/configuration/mapreduce-log4j.xml
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/OOZIE/configuration/oozie-log4j.xml
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/OOZIE/package/scripts/oozie.py
91da7ae 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/OOZIE/package/scripts/params.py
0466ad8 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/OOZIE/package/templates/oozie-log4j.properties.j2
e4a2662 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/PIG/configuration/pig-log4j.xml
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/PIG/package/scripts/params.py
86e962c 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/PIG/package/scripts/pig.py c2d7b02

  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/PIG/package/templates/log4j.properties.j2
9ef6e2c 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/ZOOKEEPER/configuration/zookeeper-log4j.xml
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/ZOOKEEPER/package/scripts/params.py
9acc0c9 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/ZOOKEEPER/package/scripts/zookeeper.py
c49eb22 
  ambari-server/src/main/resources/stacks/HDP/1.3.3/services/ZOOKEEPER/package/templates/log4j.properties.j2
db69564 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/hooks/before-START/scripts/params.py 712a5ab

  ambari-server/src/main/resources/stacks/HDP/2.1.1/hooks/before-START/scripts/shared_initialization.py
f2644aa 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/hooks/before-START/templates/log4j.properties.j2
6c02292 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HBASE/configuration/hbase-log4j.xml
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HBASE/package/scripts/hbase.py
bd33463 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HBASE/package/scripts/params.py
674b2d9 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HDFS/configuration/hdfs-log4j.xml
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HIVE/configuration/hive-exec-log4j.xml
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HIVE/configuration/hive-log4j.xml
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HIVE/package/scripts/hive.py
b37ebb2 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HIVE/package/scripts/params.py
0cf89be 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/OOZIE/configuration/oozie-log4j.xml
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/OOZIE/package/scripts/oozie.py
1422d1e 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/OOZIE/package/scripts/params.py
1ad1412 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/PIG/configuration/pig-log4j.xml
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/PIG/package/scripts/params.py
86e962c 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/PIG/package/scripts/pig.py c2d7b02

  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/PIG/package/templates/log4j.properties.j2
9ef6e2c 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/YARN/configuration/yarn-log4j.xml
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/ZOOKEEPER/configuration/zookeeper-log4j.xml
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/ZOOKEEPER/package/scripts/params.py
9acc0c9 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/ZOOKEEPER/package/scripts/zookeeper.py
c49eb22 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/ZOOKEEPER/package/templates/log4j.properties.j2
db69564 

Diff: https://reviews.apache.org/r/16965/diff/


Testing
-------

Manual testing on HDP1.3.4 and HDP2.0.8 on multinode cluster


Thanks,

Dmytro Sen


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message