ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmytro Sen" <d...@hortonworks.com>
Subject Re: Review Request 18123: Custom log4j configs should be stored as raw text
Date Fri, 14 Feb 2014 16:59:26 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18123/
-----------------------------------------------------------

(Updated Feb. 14, 2014, 4:59 p.m.)


Review request for Ambari, Dmitro Lisnichenko and Sumit Mohanty.


Changes
-------

Added unit tests for oozie, merged patch with current trunk


Bugs: AMBARI-4678
    https://issues.apache.org/jira/browse/AMBARI-4678


Repository: ambari


Description
-------

In 1.5.0, we are introducing customization of log4j.properties for various services ("hdfs-log4j"
configuration, etc).
Instead of storing the log4j configuration as a bunch of key-value pairs, we should store
the entire log4j.properties file content as a single attribute (lets call it "content") whose
value is the entire file content of log4j.properties in raw text.
This would be much easier for the end user to view and modify. Also, this allows us to retain
comments and logical breaks in the file.


Actually we could reuse current design. We just have to store the log4g propertieas not as
key-value pair, but as a raw text in a single "content" attribute.

Steps to be completed to put log4.
1. UI downloads from the server log4j.properties as a raw text
2. User is able to customize this log4j configuration in textarea
3. UI escapes quotes(\"), new lines(\n), other special characters and puts the config like:
{noformat}
curl  -H 'X-Requested-By: X-Requested-By' -u admin:admin -i -X PUT -d '{"Clusters":{"desired_configs":{"type":"oozie-log4j","tag":"version1","properties":{"content":"line1\nline2\n"}}}}'
http://localhost:8080/api/v1/clusters/c1
{noformat}
4. Ambari-agent stores the "content" attribute value to the spesified log4j.properties file
using resource File like :
{code}
  File(format("{params.conf_dir}/oozie-log4j.properties"),
         mode=0644,
         group=params.user_group,
         owner=params.oozie_user,
         content=params.configuration['oozie-log4j']['content']
  )
{code}

I've tested this design. It allows to store and deliver to the agent log4j.properties as a
raw text with any comments or escaped characters.


Diffs (updated)
-----

  ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 0b62c87

  ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/shared_initialization.py
d6521ee 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HBASE/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HBASE/package/scripts/hbase.py
c140a6d 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HBASE/package/scripts/params.py
9722034 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/configuration/hive-exec-log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/configuration/hive-log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/package/scripts/hive.py
940f8c3 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/package/scripts/params.py
b999aec 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/OOZIE/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/OOZIE/package/scripts/oozie.py
151a1f8 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/OOZIE/package/scripts/params.py
517ee73 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/PIG/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/PIG/package/scripts/params.py
3cd60d2 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/PIG/package/scripts/pig.py 0450d5e

  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/ZOOKEEPER/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/ZOOKEEPER/configuration/zookeeper-log4j.xml
8fb48b9 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/ZOOKEEPER/package/scripts/params.py
60fa5da 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/ZOOKEEPER/package/scripts/zookeeper.py
7b74b4c 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 3239c7a

  ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py
2f2ca8b 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase.py
d899d17 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/params.py
a9086e7 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/configuration/hive-exec-log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/configuration/hive-log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/hive.py
bb1c065 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/params.py
ea51420 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/package/scripts/oozie.py
2efe535 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/package/scripts/params.py
dc52b39 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/PIG/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/PIG/package/scripts/params.py
3cd60d2 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/PIG/package/scripts/pig.py d8aadf7

  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/configuration/zookeeper-log4j.xml
8fb48b9 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/package/scripts/params.py
60fa5da 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/package/scripts/zookeeper.py
ac69829 
  ambari-server/src/test/python/stacks/1.3.2/HBASE/test_hbase_client.py 5ef19c3 
  ambari-server/src/test/python/stacks/1.3.2/HBASE/test_hbase_master.py 1d1baca 
  ambari-server/src/test/python/stacks/1.3.2/HBASE/test_hbase_regionserver.py a031b04 
  ambari-server/src/test/python/stacks/1.3.2/HIVE/test_hive_client.py d1610d6 
  ambari-server/src/test/python/stacks/1.3.2/HIVE/test_hive_metastore.py c970cc8 
  ambari-server/src/test/python/stacks/1.3.2/HIVE/test_hive_server.py 45b7a0a 
  ambari-server/src/test/python/stacks/1.3.2/OOZIE/test_oozie_client.py e60b1ad 
  ambari-server/src/test/python/stacks/1.3.2/OOZIE/test_oozie_server.py 7093ac7 
  ambari-server/src/test/python/stacks/1.3.2/PIG/test_pig_client.py 1856be6 
  ambari-server/src/test/python/stacks/1.3.2/ZOOKEEPER/test_zookeeper_client.py f7de9f9 
  ambari-server/src/test/python/stacks/1.3.2/ZOOKEEPER/test_zookeeper_server.py 81df5e7 
  ambari-server/src/test/python/stacks/1.3.2/configs/default.json 653be3c 
  ambari-server/src/test/python/stacks/1.3.2/configs/secured.json e711ff2 
  ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py d3b7788

  ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_client.py bd3228f 
  ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py ef58f17 
  ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_regionserver.py 3efe21e 
  ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_client.py ec3c895 
  ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_metastore.py c08a260 
  ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 071d108 
  ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_client.py 358cab9 
  ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 8e35e23 
  ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_client.py 0143e27 
  ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_client.py db1b883 
  ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_server.py f5ac3eb 
  ambari-server/src/test/python/stacks/2.0.6/configs/default.json d3d5ba3 
  ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 61a4830 
  ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 6045f39


Diff: https://reviews.apache.org/r/18123/diff/


Testing (updated)
-------

Fixed unit tests
[INFO] Reactor Summary:
[INFO] 
[INFO] Ambari Main ....................................... SUCCESS [0.069s]
[INFO] Apache Ambari Project POM ......................... SUCCESS [0.026s]
[INFO] Ambari Web ........................................ SUCCESS [11.989s]
[INFO] Ambari Views ...................................... SUCCESS [1.771s]
[INFO] Ambari Server ..................................... SUCCESS [11:08.445s]
[INFO] Ambari Agent ...................................... SUCCESS [12.989s]
[INFO] Ambari Client ..................................... SUCCESS [0.511s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11:38.352s
[INFO] Finished at: Fri Feb 14 19:52:25 FET 2014
[INFO] Final Memory: 159M/932M


Thanks,

Dmytro Sen


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message