ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Onischuk" <aonis...@hortonworks.com>
Subject Re: Review Request 16965: Allow log4j properties to be applied via the API in Ambari for hadoop/oozie/hbase/hive/zookeeper/pig
Date Thu, 23 Jan 2014 11:55:42 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16965/#review32611
-----------------------------------------------------------


property_map (rca_property_map) should be put into params, not into conreate script. Not to
mess up parameter constans with code. Also then this part whould go to params too:
{code}	
    if params.rca_enabled:
      params.log4j_props.update(property_map)
{code}

Also, if we don't have some log4j properties map sent from server, you are just changing the
permission for file. Are all that files installed with the packages, or why you do so?

- Andrew Onischuk


On Jan. 23, 2014, 1:28 a.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16965/
> -----------------------------------------------------------
> 
> (Updated Jan. 23, 2014, 1:28 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Oleksandr Diachenko, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-3537
>     https://issues.apache.org/jira/browse/AMBARI-3537
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 
> New cluster deployment scenario:
> - During cluster deployment UI can read default values from "....\resources\stacks\HDP\2.0.8\services\HDFS\configuration\hdfs-log4j.xml",
but user is able to modify them before deployment.
> - UI makes calls like
>      curl -i -X PUT -d '{"Clusters":{"desired_configs":{"type":"hdfs-log4j","tag":"version1","properties":{"hadoop.root.logger":"INFO,console"}}}}'
http://localhost:8080/api/v1/clusters/c1
> - python scripts at the agent side store hdfs-log4j configs as log4j.properties in /etc/hadoop/conf/
> - log4j.properties management is made by Ambari UI or API
> 
> Cluster upgrade scenario(not implemented yet):
> - before running "ambari-server upgrade" command, user modifies *-log4j.xml files at
the upgrade stack directory (like stacks\HDP\2.0.8\services\HDFS\configuration)
> - run ambari-server upgrade
> 
> 
> After applying the patch, ambari server is able to manage log4j properties at the files
below:
> /etc/hadoop/conf/log4j.properties
> /etc/hbase/conf/log4j.properties
> /etc/hive/conf/hive-log4j.properties.template
> /etc/hive/conf/hive-log4j.properties.template
> /etc/oozie/conf/oozie-log4j.properties
> /etc/pig/conf/log4j.properties
> /etc/zookeeper/conf/log4j.properties
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/resource_management/libraries/providers/__init__.py c5e7bae

>   ambari-agent/src/main/python/resource_management/libraries/providers/properties_file.py
PRE-CREATION 
>   ambari-agent/src/main/python/resource_management/libraries/resources/__init__.py bad8470

>   ambari-agent/src/main/python/resource_management/libraries/resources/properties_file.py
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/hooks/before-START/scripts/params.py
aabb406 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/hooks/before-START/scripts/shared_initialization.py
7b406e1 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/hooks/before-START/templates/log4j.properties.j2
577ad04 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HBASE/configuration/hbase-log4j.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HBASE/package/scripts/hbase.py
bd33463 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HBASE/package/scripts/params.py
95880cb 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HDFS/configuration/hdfs-log4j.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HIVE/configuration/hive-exec-log4j.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HIVE/configuration/hive-log4j.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HIVE/package/scripts/hive.py
b37ebb2 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HIVE/package/scripts/params.py
0cf89be 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/MAPREDUCE/configuration/mapreduce-log4j.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/OOZIE/configuration/oozie-log4j.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/OOZIE/package/scripts/oozie.py
91da7ae 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/OOZIE/package/scripts/params.py
0466ad8 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/OOZIE/package/templates/oozie-log4j.properties.j2
e4a2662 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/PIG/configuration/pig-log4j.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/PIG/package/scripts/params.py
86e962c 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/PIG/package/scripts/pig.py
c2d7b02 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/PIG/package/templates/log4j.properties.j2
9ef6e2c 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/ZOOKEEPER/configuration/zookeeper-log4j.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/ZOOKEEPER/package/scripts/params.py
9acc0c9 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/ZOOKEEPER/package/scripts/zookeeper.py
c49eb22 
>   ambari-server/src/main/resources/stacks/HDP/1.3.3/services/ZOOKEEPER/package/templates/log4j.properties.j2
db69564 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/hooks/before-START/scripts/params.py
712a5ab 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/hooks/before-START/scripts/shared_initialization.py
f2644aa 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/hooks/before-START/templates/log4j.properties.j2
6c02292 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HBASE/configuration/hbase-log4j.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HBASE/package/scripts/hbase.py
bd33463 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HBASE/package/scripts/params.py
674b2d9 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HDFS/configuration/hdfs-log4j.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HIVE/configuration/hive-exec-log4j.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HIVE/configuration/hive-log4j.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HIVE/package/scripts/hive.py
b37ebb2 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HIVE/package/scripts/params.py
0cf89be 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/OOZIE/configuration/oozie-log4j.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/OOZIE/package/scripts/oozie.py
1422d1e 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/OOZIE/package/scripts/params.py
1ad1412 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/PIG/configuration/pig-log4j.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/PIG/package/scripts/params.py
86e962c 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/PIG/package/scripts/pig.py
c2d7b02 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/PIG/package/templates/log4j.properties.j2
9ef6e2c 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/YARN/configuration/yarn-log4j.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/ZOOKEEPER/configuration/zookeeper-log4j.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/ZOOKEEPER/package/scripts/params.py
9acc0c9 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/ZOOKEEPER/package/scripts/zookeeper.py
c49eb22 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/ZOOKEEPER/package/templates/log4j.properties.j2
db69564 
> 
> Diff: https://reviews.apache.org/r/16965/diff/
> 
> 
> Testing
> -------
> 
> Manual testing on HDP1.3.4 and HDP2.0.8 on multinode cluster
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message