ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmytro Sen" <d...@hortonworks.com>
Subject Re: Review Request 18123: Custom log4j configs should be stored as raw text
Date Fri, 21 Feb 2014 10:17:34 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18123/
-----------------------------------------------------------

(Updated Feb. 21, 2014, 10:17 a.m.)


Review request for Ambari, Dmitro Lisnichenko and Sumit Mohanty.


Changes
-------

Updated the patch, new it's may be applied  to the trunk


Bugs: AMBARI-4678
    https://issues.apache.org/jira/browse/AMBARI-4678


Repository: ambari


Description
-------

In 1.5.0, we are introducing customization of log4j.properties for various services ("hdfs-log4j"
configuration, etc).
Instead of storing the log4j configuration as a bunch of key-value pairs, we should store
the entire log4j.properties file content as a single attribute (lets call it "content") whose
value is the entire file content of log4j.properties in raw text.
This would be much easier for the end user to view and modify. Also, this allows us to retain
comments and logical breaks in the file.


Actually we could reuse current design. We just have to store the log4g propertieas not as
key-value pair, but as a raw text in a single "content" attribute.

Steps to be completed to put log4.
1. UI downloads from the server log4j.properties as a raw text
2. User is able to customize this log4j configuration in textarea
3. UI escapes quotes(\"), new lines(\n), other special characters and puts the config like:
{noformat}
curl  -H 'X-Requested-By: X-Requested-By' -u admin:admin -i -X PUT -d '{"Clusters":{"desired_configs":{"type":"oozie-log4j","tag":"version1","properties":{"content":"line1\nline2\n"}}}}'
http://localhost:8080/api/v1/clusters/c1
{noformat}
4. Ambari-agent stores the "content" attribute value to the spesified log4j.properties file
using resource File like :
{code}
  File(format("{params.conf_dir}/oozie-log4j.properties"),
         mode=0644,
         group=params.user_group,
         owner=params.oozie_user,
         content=params.configuration['oozie-log4j']['content']
  )
{code}

I've tested this design. It allows to store and deliver to the agent log4j.properties as a
raw text with any comments or escaped characters.


Diffs (updated)
-----

  ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py e602895

  ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/shared_initialization.py
d6521ee 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HBASE/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HBASE/package/scripts/hbase.py
c140a6d 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HBASE/package/scripts/params.py
9722034 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/configuration/hive-exec-log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/configuration/hive-log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/package/scripts/hive.py
38fb498 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/package/scripts/params.py
b999aec 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/OOZIE/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/OOZIE/package/scripts/oozie.py
151a1f8 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/OOZIE/package/scripts/params.py
517ee73 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/PIG/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/PIG/package/scripts/params.py
3cd60d2 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/PIG/package/scripts/pig.py 0450d5e

  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/ZOOKEEPER/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/ZOOKEEPER/package/scripts/params.py
60fa5da 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/ZOOKEEPER/package/scripts/zookeeper.py
7b74b4c 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 85ee2b9

  ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py
2f2ca8b 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase.py
d899d17 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/params.py
a9086e7 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/configuration/hive-exec-log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/configuration/hive-log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/hive.py
94b0a45 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/params.py
d201832 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/package/scripts/oozie.py
2efe535 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/package/scripts/params.py
dc52b39 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/PIG/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/PIG/package/scripts/params.py
3cd60d2 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/PIG/package/scripts/pig.py d8aadf7

  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/package/scripts/params.py
60fa5da 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/package/scripts/zookeeper.py
ac69829 
  ambari-server/src/test/python/stacks/1.3.2/HBASE/test_hbase_client.py 5ef19c3 
  ambari-server/src/test/python/stacks/1.3.2/HBASE/test_hbase_master.py 1d1baca 
  ambari-server/src/test/python/stacks/1.3.2/HBASE/test_hbase_regionserver.py a031b04 
  ambari-server/src/test/python/stacks/1.3.2/HIVE/test_hive_client.py d1610d6 
  ambari-server/src/test/python/stacks/1.3.2/HIVE/test_hive_metastore.py 01bd904 
  ambari-server/src/test/python/stacks/1.3.2/HIVE/test_hive_server.py 9bedc4d 
  ambari-server/src/test/python/stacks/1.3.2/OOZIE/test_oozie_client.py eed555d 
  ambari-server/src/test/python/stacks/1.3.2/OOZIE/test_oozie_server.py b70dda3 
  ambari-server/src/test/python/stacks/1.3.2/PIG/test_pig_client.py 1856be6 
  ambari-server/src/test/python/stacks/1.3.2/ZOOKEEPER/test_zookeeper_client.py f7de9f9 
  ambari-server/src/test/python/stacks/1.3.2/ZOOKEEPER/test_zookeeper_server.py 81df5e7 
  ambari-server/src/test/python/stacks/1.3.2/configs/default.json 70a3132 
  ambari-server/src/test/python/stacks/1.3.2/configs/secured.json 7e3d39d 
  ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py d8643d6

  ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_client.py bd3228f 
  ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py ef58f17 
  ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_regionserver.py 3efe21e 
  ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_client.py ec3c895 
  ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_metastore.py aa8037b 
  ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 73c6003 
  ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_client.py bc84440 
  ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 6b438a6 
  ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_client.py 0143e27 
  ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_client.py db1b883 
  ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_server.py f5ac3eb 
  ambari-server/src/test/python/stacks/2.0.6/configs/default.json 719c2e9 
  ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 758fe0b 
  ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 159ba3e


Diff: https://reviews.apache.org/r/18123/diff/


Testing
-------

Fixed unit tests
[INFO] Reactor Summary:
[INFO] 
[INFO] Ambari Main ....................................... SUCCESS [0.069s]
[INFO] Apache Ambari Project POM ......................... SUCCESS [0.026s]
[INFO] Ambari Web ........................................ SUCCESS [11.989s]
[INFO] Ambari Views ...................................... SUCCESS [1.771s]
[INFO] Ambari Server ..................................... SUCCESS [11:08.445s]
[INFO] Ambari Agent ...................................... SUCCESS [12.989s]
[INFO] Ambari Client ..................................... SUCCESS [0.511s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11:38.352s
[INFO] Finished at: Fri Feb 14 19:52:25 FET 2014
[INFO] Final Memory: 159M/932M


Thanks,

Dmytro Sen


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message