Return-Path: X-Original-To: apmail-ambari-dev-archive@www.apache.org Delivered-To: apmail-ambari-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8C4AA10B77 for ; Fri, 14 Feb 2014 16:17:43 +0000 (UTC) Received: (qmail 11976 invoked by uid 500); 14 Feb 2014 16:17:42 -0000 Delivered-To: apmail-ambari-dev-archive@ambari.apache.org Received: (qmail 11937 invoked by uid 500); 14 Feb 2014 16:17:42 -0000 Mailing-List: contact dev-help@ambari.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ambari.apache.org Delivered-To: mailing list dev@ambari.apache.org Received: (qmail 11924 invoked by uid 99); 14 Feb 2014 16:17:41 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Feb 2014 16:17:41 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id 268CC1C0429; Fri, 14 Feb 2014 16:17:40 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============0543339296178920422==" MIME-Version: 1.0 Subject: Review Request 18123: Custom log4j configs should be stored as raw text From: "Dmytro Sen" To: "Sumit Mohanty" , "Dmitro Lisnichenko" Cc: "Dmytro Sen" , "Ambari" Date: Fri, 14 Feb 2014 16:17:40 -0000 Message-ID: <20140214161740.29299.39312@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org Auto-Submitted: auto-generated Sender: "Dmytro Sen" X-ReviewGroup: Ambari X-ReviewRequest-URL: https://reviews.apache.org/r/18123/ X-Sender: "Dmytro Sen" Reply-To: "Dmytro Sen" X-ReviewRequest-Repository: ambari --===============0543339296178920422== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18123/ ----------------------------------------------------------- Review request for Ambari, Dmitro Lisnichenko and Sumit Mohanty. Bugs: AMBARI-4678 https://issues.apache.org/jira/browse/AMBARI-4678 Repository: ambari Description ------- In 1.5.0, we are introducing customization of log4j.properties for various services ("hdfs-log4j" configuration, etc). Instead of storing the log4j configuration as a bunch of key-value pairs, we should store the entire log4j.properties file content as a single attribute (lets call it "content") whose value is the entire file content of log4j.properties in raw text. This would be much easier for the end user to view and modify. Also, this allows us to retain comments and logical breaks in the file. Actually we could reuse current design. We just have to store the log4g propertieas not as key-value pair, but as a raw text in a single "content" attribute. Steps to be completed to put log4. 1. UI downloads from the server log4j.properties as a raw text 2. User is able to customize this log4j configuration in textarea 3. UI escapes quotes(\"), new lines(\n), other special characters and puts the config like: {noformat} curl -H 'X-Requested-By: X-Requested-By' -u admin:admin -i -X PUT -d '{"Clusters":{"desired_configs":{"type":"oozie-log4j","tag":"version1","properties":{"content":"line1\nline2\n"}}}}' http://localhost:8080/api/v1/clusters/c1 {noformat} 4. Ambari-agent stores the "content" attribute value to the spesified log4j.properties file using resource File like : {code} File(format("{params.conf_dir}/oozie-log4j.properties"), mode=0644, group=params.user_group, owner=params.oozie_user, content=params.configuration['oozie-log4j']['content'] ) {code} I've tested this design. It allows to store and deliver to the agent log4j.properties as a raw text with any comments or escaped characters. Diffs ----- ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 211c2bb ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/shared_initialization.py d6521ee ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HBASE/configuration/log4j.properties PRE-CREATION ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HBASE/package/scripts/hbase.py c140a6d ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HBASE/package/scripts/params.py 923b6ce ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/configuration/log4j.properties PRE-CREATION ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/configuration/hive-exec-log4j.properties PRE-CREATION ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/configuration/hive-log4j.properties PRE-CREATION ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/package/scripts/hive.py 940f8c3 ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/package/scripts/params.py a9d3e9c ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/configuration/log4j.properties PRE-CREATION ambari-server/src/main/resources/stacks/HDP/1.3.2/services/OOZIE/configuration/log4j.properties PRE-CREATION ambari-server/src/main/resources/stacks/HDP/1.3.2/services/OOZIE/package/scripts/oozie.py 151a1f8 ambari-server/src/main/resources/stacks/HDP/1.3.2/services/OOZIE/package/scripts/params.py 93f3dd1 ambari-server/src/main/resources/stacks/HDP/1.3.2/services/PIG/configuration/log4j.properties PRE-CREATION ambari-server/src/main/resources/stacks/HDP/1.3.2/services/PIG/package/scripts/params.py 636f882 ambari-server/src/main/resources/stacks/HDP/1.3.2/services/PIG/package/scripts/pig.py 0450d5e ambari-server/src/main/resources/stacks/HDP/1.3.2/services/ZOOKEEPER/configuration/log4j.properties PRE-CREATION ambari-server/src/main/resources/stacks/HDP/1.3.2/services/ZOOKEEPER/package/scripts/params.py 3c02248 ambari-server/src/main/resources/stacks/HDP/1.3.2/services/ZOOKEEPER/package/scripts/zookeeper.py 7b74b4c ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 1955a3c ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py 2f2ca8b ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/configuration/log4j.properties PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase.py d899d17 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/params.py 0422a7c ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/configuration/log4j.properties PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/configuration/hive-exec-log4j.properties PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/configuration/hive-log4j.properties PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/hive.py bb1c065 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/params.py 7ed4322 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/configuration/log4j.properties PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/package/scripts/oozie.py 2efe535 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/package/scripts/params.py 463dccc ambari-server/src/main/resources/stacks/HDP/2.0.6/services/PIG/configuration/log4j.properties PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.0.6/services/PIG/package/scripts/params.py 636f882 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/PIG/package/scripts/pig.py d8aadf7 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/configuration/log4j.properties PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/configuration/log4j.properties PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/package/scripts/params.py 3c02248 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/package/scripts/zookeeper.py ac69829 ambari-server/src/test/python/stacks/1.3.2/HBASE/test_hbase_client.py 5ef19c3 ambari-server/src/test/python/stacks/1.3.2/HBASE/test_hbase_master.py 1d1baca ambari-server/src/test/python/stacks/1.3.2/HBASE/test_hbase_regionserver.py a031b04 ambari-server/src/test/python/stacks/1.3.2/HIVE/test_hive_client.py d1610d6 ambari-server/src/test/python/stacks/1.3.2/HIVE/test_hive_metastore.py c970cc8 ambari-server/src/test/python/stacks/1.3.2/HIVE/test_hive_server.py 45b7a0a ambari-server/src/test/python/stacks/1.3.2/PIG/test_pig_client.py bdb0a2f ambari-server/src/test/python/stacks/1.3.2/ZOOKEEPER/test_zookeeper_client.py f7de9f9 ambari-server/src/test/python/stacks/1.3.2/ZOOKEEPER/test_zookeeper_server.py 81df5e7 ambari-server/src/test/python/stacks/1.3.2/configs/default.json 9e087b7 ambari-server/src/test/python/stacks/1.3.2/configs/secured.json a5ee2fd ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py d3b7788 ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_client.py bd3228f ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py ef58f17 ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_regionserver.py 3efe21e ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_client.py ec3c895 ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_metastore.py c08a260 ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 071d108 ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_client.py 810d6df ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_client.py db1b883 ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_server.py f5ac3eb ambari-server/src/test/python/stacks/2.0.6/configs/default.json c96a303 ambari-server/src/test/python/stacks/2.0.6/configs/secured.json c4776bc ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 6045f39 Diff: https://reviews.apache.org/r/18123/diff/ Testing ------- Fixed unit tests Thanks, Dmytro Sen --===============0543339296178920422==--