ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmytro Sen" <d...@hortonworks.com>
Subject Review Request 18123: Custom log4j configs should be stored as raw text
Date Fri, 14 Feb 2014 16:17:40 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18123/
-----------------------------------------------------------

Review request for Ambari, Dmitro Lisnichenko and Sumit Mohanty.


Bugs: AMBARI-4678
    https://issues.apache.org/jira/browse/AMBARI-4678


Repository: ambari


Description
-------

In 1.5.0, we are introducing customization of log4j.properties for various services ("hdfs-log4j"
configuration, etc).
Instead of storing the log4j configuration as a bunch of key-value pairs, we should store
the entire log4j.properties file content as a single attribute (lets call it "content") whose
value is the entire file content of log4j.properties in raw text.
This would be much easier for the end user to view and modify. Also, this allows us to retain
comments and logical breaks in the file.


Actually we could reuse current design. We just have to store the log4g propertieas not as
key-value pair, but as a raw text in a single "content" attribute.

Steps to be completed to put log4.
1. UI downloads from the server log4j.properties as a raw text
2. User is able to customize this log4j configuration in textarea
3. UI escapes quotes(\"), new lines(\n), other special characters and puts the config like:
{noformat}
curl  -H 'X-Requested-By: X-Requested-By' -u admin:admin -i -X PUT -d '{"Clusters":{"desired_configs":{"type":"oozie-log4j","tag":"version1","properties":{"content":"line1\nline2\n"}}}}'
http://localhost:8080/api/v1/clusters/c1
{noformat}
4. Ambari-agent stores the "content" attribute value to the spesified log4j.properties file
using resource File like :
{code}
  File(format("{params.conf_dir}/oozie-log4j.properties"),
         mode=0644,
         group=params.user_group,
         owner=params.oozie_user,
         content=params.configuration['oozie-log4j']['content']
  )
{code}

I've tested this design. It allows to store and deliver to the agent log4j.properties as a
raw text with any comments or escaped characters.


Diffs
-----

  ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 211c2bb

  ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/shared_initialization.py
d6521ee 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HBASE/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HBASE/package/scripts/hbase.py
c140a6d 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HBASE/package/scripts/params.py
923b6ce 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/configuration/hive-exec-log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/configuration/hive-log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/package/scripts/hive.py
940f8c3 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/package/scripts/params.py
a9d3e9c 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/OOZIE/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/OOZIE/package/scripts/oozie.py
151a1f8 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/OOZIE/package/scripts/params.py
93f3dd1 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/PIG/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/PIG/package/scripts/params.py
636f882 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/PIG/package/scripts/pig.py 0450d5e

  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/ZOOKEEPER/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/ZOOKEEPER/package/scripts/params.py
3c02248 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/ZOOKEEPER/package/scripts/zookeeper.py
7b74b4c 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 1955a3c

  ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py
2f2ca8b 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase.py
d899d17 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/params.py
0422a7c 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/configuration/hive-exec-log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/configuration/hive-log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/hive.py
bb1c065 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/params.py
7ed4322 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/package/scripts/oozie.py
2efe535 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/package/scripts/params.py
463dccc 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/PIG/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/PIG/package/scripts/params.py
636f882 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/PIG/package/scripts/pig.py d8aadf7

  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/configuration/log4j.properties
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/package/scripts/params.py
3c02248 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/package/scripts/zookeeper.py
ac69829 
  ambari-server/src/test/python/stacks/1.3.2/HBASE/test_hbase_client.py 5ef19c3 
  ambari-server/src/test/python/stacks/1.3.2/HBASE/test_hbase_master.py 1d1baca 
  ambari-server/src/test/python/stacks/1.3.2/HBASE/test_hbase_regionserver.py a031b04 
  ambari-server/src/test/python/stacks/1.3.2/HIVE/test_hive_client.py d1610d6 
  ambari-server/src/test/python/stacks/1.3.2/HIVE/test_hive_metastore.py c970cc8 
  ambari-server/src/test/python/stacks/1.3.2/HIVE/test_hive_server.py 45b7a0a 
  ambari-server/src/test/python/stacks/1.3.2/PIG/test_pig_client.py bdb0a2f 
  ambari-server/src/test/python/stacks/1.3.2/ZOOKEEPER/test_zookeeper_client.py f7de9f9 
  ambari-server/src/test/python/stacks/1.3.2/ZOOKEEPER/test_zookeeper_server.py 81df5e7 
  ambari-server/src/test/python/stacks/1.3.2/configs/default.json 9e087b7 
  ambari-server/src/test/python/stacks/1.3.2/configs/secured.json a5ee2fd 
  ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py d3b7788

  ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_client.py bd3228f 
  ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py ef58f17 
  ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_regionserver.py 3efe21e 
  ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_client.py ec3c895 
  ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_metastore.py c08a260 
  ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 071d108 
  ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_client.py 810d6df 
  ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_client.py db1b883 
  ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_server.py f5ac3eb 
  ambari-server/src/test/python/stacks/2.0.6/configs/default.json c96a303 
  ambari-server/src/test/python/stacks/2.0.6/configs/secured.json c4776bc 
  ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 6045f39


Diff: https://reviews.apache.org/r/18123/diff/


Testing
-------

Fixed unit tests


Thanks,

Dmytro Sen


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message