Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 1F9AC200B0F for ; Thu, 2 Jun 2016 17:02:39 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 1E283160A54; Thu, 2 Jun 2016 15:02:39 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 48F4A160A55 for ; Thu, 2 Jun 2016 17:02:35 +0200 (CEST) Received: (qmail 62984 invoked by uid 500); 2 Jun 2016 15:02:34 -0000 Mailing-List: contact commits-help@ambari.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: ambari-dev@ambari.apache.org Delivered-To: mailing list commits@ambari.apache.org Received: (qmail 60824 invoked by uid 99); 2 Jun 2016 15:02:32 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 02 Jun 2016 15:02:32 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id C45BDE93D8; Thu, 2 Jun 2016 15:02:31 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: dmitriusan@apache.org To: commits@ambari.apache.org Date: Thu, 02 Jun 2016 15:02:54 -0000 Message-Id: In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [24/47] ambari git commit: AMBARI-16272. Ambari Upgrade shouldn't automatically add stack configs (dlysnichenko) archived-at: Thu, 02 Jun 2016 15:02:39 -0000 http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/YARN/configuration/yarn-site.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/YARN/configuration/yarn-site.xml b/ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/YARN/configuration/yarn-site.xml index d562246..9777ee4 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/YARN/configuration/yarn-site.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/YARN/configuration/yarn-site.xml @@ -16,15 +16,14 @@ See the License for the specific language governing permissions and limitations under the License. --> - - - - + yarn.application.classpath $HADOOP_CONF_DIR,/usr/hdp/current/hadoop-client/*,/usr/hdp/current/hadoop-client/lib/*,/usr/hdp/current/hadoop-hdfs-client/*,/usr/hdp/current/hadoop-hdfs-client/lib/*,/usr/hdp/current/hadoop-yarn-client/*,/usr/hdp/current/hadoop-yarn-client/lib/* Classpath for typical applications. + + hadoop.registry.rm.enabled @@ -32,6 +31,8 @@ Is the registry enabled: does the RM start it up, create the user and system paths, and purge service records when containers, application attempts and applications complete + + hadoop.registry.zk.quorum @@ -39,11 +40,15 @@ List of hostname:port pairs defining the zookeeper quorum binding for the registry + + yarn.nodemanager.recovery.enabled true Enable the node manager to recover after starting + + yarn.nodemanager.recovery.dir @@ -52,16 +57,22 @@ The local filesystem directory in which the node manager will store state when recovery is enabled. + + yarn.client.nodemanager-connect.retry-interval-ms 10000 Time interval between each attempt to connect to NM + + yarn.client.nodemanager-connect.max-wait-ms 60000 Max time to wait to establish a connection to NM + + yarn.resourcemanager.recovery.enabled @@ -70,6 +81,8 @@ Enable RM to recover state after starting. If true, then yarn.resourcemanager.store.class must be specified. + + yarn.resourcemanager.work-preserving-recovery.enabled @@ -77,6 +90,8 @@ Enable RM work preserving recovery. This configuration is private to YARN for experimenting the feature. + + yarn.resourcemanager.store.class @@ -87,6 +102,8 @@ the store is implicitly fenced; meaning a single ResourceManager is able to use the store at any point in time. + + yarn.resourcemanager.zk-address @@ -94,31 +111,43 @@ List Host:Port of the ZooKeeper servers to be used by the RM. comma separated host:port pairs, each corresponding to a zk server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If the optional chroot suffix is used the example would look like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a" where the client would be rooted at "/app/a" and all paths would be relative to this root - ie getting/setting/etc... "/foo/bar" would result in operations being run on "/app/a/foo/bar" (from the server perspective). + + yarn.resourcemanager.zk-state-store.parent-path /rmstore Full path of the ZooKeeper znode where RM state will be stored. This must be supplied when using org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore as the value for yarn.resourcemanager.store.class + + yarn.resourcemanager.zk-acl world:anyone:rwcda ACL's to be used for ZooKeeper znodes. + + yarn.resourcemanager.work-preserving-recovery.scheduling-wait-ms 10000 Set the amount of time RM waits before allocating new containers on work-preserving-recovery. Such wait period gives RM a chance to settle down resyncing with NMs in the cluster on recovery, before assigning new containers to applications. + + yarn.resourcemanager.connect.retry-interval.ms 30000 How often to try connecting to the ResourceManager. + + yarn.resourcemanager.connect.max-wait.ms 900000 Maximum time to wait to establish connection to ResourceManager + + yarn.resourcemanager.zk-retry-interval-ms @@ -128,67 +157,91 @@ automatically from yarn.resourcemanager.zk-timeout-ms and yarn.resourcemanager.zk-num-retries." + + yarn.resourcemanager.zk-num-retries 1000 Number of times RM tries to connect to ZooKeeper. + + yarn.resourcemanager.zk-timeout-ms 10000 ZooKeeper session timeout in milliseconds. Session expiration is managed by the ZooKeeper cluster itself, not by the client. This value is used by the cluster to determine when the client's session expires. Expirations happens when the cluster does not hear from the client within the specified session timeout period (i.e. no heartbeat). + + yarn.resourcemanager.state-store.max-completed-applications ${yarn.resourcemanager.max-completed-applications} The maximum number of completed applications RM state store keeps, less than or equals to ${yarn.resourcemanager.max-completed-applications}. By default, it equals to ${yarn.resourcemanager.max-completed-applications}. This ensures that the applications kept in the state store are consistent with the applications remembered in RM memory. Any values larger than ${yarn.resourcemanager.max-completed-applications} will be reset to ${yarn.resourcemanager.max-completed-applications}. Note that this value impacts the RM recovery performance.Typically, a smaller value indicates better performance on RM recovery. + + yarn.resourcemanager.fs.state-store.retry-policy-spec 2000, 500 hdfs client retry policy specification. hdfs client retry is always enabled. Specified in pairs of sleep-time and number-of-retries and (t0, n0), (t1, n1), ..., the first n0 retries sleep t0 milliseconds on average, the following n1 retries sleep t1 milliseconds on average, and so on. + + yarn.resourcemanager.fs.state-store.uri RI pointing to the location of the FileSystem path where RM state will be stored. This must be supplied when using org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore as the value for yarn.resourcemanager.store.class + + yarn.resourcemanager.ha.enabled false enable RM HA or not + + yarn.nodemanager.linux-container-executor.resources-handler.class org.apache.hadoop.yarn.server.nodemanager.util.DefaultLCEResourcesHandler Pre-requisite to use CGroups + + yarn.nodemanager.linux-container-executor.cgroups.hierarchy hadoop-yarn Name of the Cgroups hierarchy under which all YARN jobs will be launched + + yarn.nodemanager.linux-container-executor.cgroups.mount false If true, YARN will automount the CGroup, however the directory needs to already exist; else, the cgroup should be mounted by the admin + + yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage false Strictly limit CPU resource usage to allocated usage even if spare CPU is available + + yarn.nodemanager.resource.cpu-vcores 8 - +

Total NM CPU vCores available to Containers

int 0 32 + + yarn.nodemanager.resource.percentage-physical-cpu-limit @@ -200,31 +253,43 @@ 0 100 + + yarn.node-labels.manager-class org.apache.hadoop.yarn.server.resourcemanager.nodelabels.MemoryRMNodeLabelsManager If user want to enable this feature, specify it to "org.apache.hadoop.yarn.server.resourcemanager.nodelabels.RMNodeLabelsManager + + yarn.node-labels.fs-store.retry-policy-spec 2000, 500 - + + + yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb 1000 This is related to disk size on the machines, admins should set one of yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb or yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage but not both. If both are set, the more conservative value will be used + + yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage 90 This is related to disk size on the machines, admins should set one of yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb or yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage but not both. If both are set, the more conservative value will be used + + yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds -1 Defines how often NMs wake up to upload log files. The default value is -1. By default, the logs will be uploaded whenthe application is finished. By setting this configure, logs can be uploaded periodically when the application is running. The minimum rolling-interval-seconds can be set is 3600. + + yarn.nodemanager.log-aggregation.debug-enabled @@ -233,31 +298,43 @@ This configuration is for debug and test purpose. By setting this configuration as true. We can break the lower bound of yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds + + yarn.nodemanager.log-aggregation.num-log-files-per-app 30 This is temporary solution. The configuration will be deleted once, we find a more scalable method to only write a single log file per LRS. + + yarn.resourcemanager.system-metrics-publisher.enabled true - + + + yarn.resourcemanager.system-metrics-publisher.dispatcher.pool-size 10 - + + + yarn.timeline-service.client.max-retries 30 - + + + yarn.timeline-service.client.retry-interval-ms 1000 - + + + yarn.timeline-service.ttl-enable @@ -265,11 +342,15 @@ Enable age off of timeline store data. + + yarn.timeline-service.leveldb-timeline-store.path /hadoop/yarn/timeline Store file name for leveldb timeline store. + + yarn.timeline-service.leveldb-timeline-store.read-cache-size @@ -277,6 +358,8 @@ Size of read cache for uncompressed blocks for leveldb timeline store in bytes. + + yarn.timeline-service.leveldb-timeline-store.start-time-read-cache-size @@ -284,6 +367,8 @@ Size of cache for recently read entity start times for leveldb timeline store in number of entities. + + yarn.timeline-service.leveldb-timeline-store.start-time-write-cache-size @@ -291,6 +376,8 @@ Size of cache for recently written entity start times for leveldb timeline store in number of entities. + + yarn.timeline-service.http-authentication.type @@ -299,11 +386,15 @@ Defines authentication used for the Timeline Server HTTP endpoint. Supported values are: simple | kerberos | $AUTHENTICATION_HANDLER_CLASSNAME + + yarn.timeline-service.http-authentication.simple.anonymous.allowed true - + + + yarn.resourcemanager.webapp.delegation-token-auth-filter.enabled @@ -314,31 +405,41 @@ tokens(fallback to kerberos if the tokens are missing). Only applicable when the http authentication type is kerberos. + + yarn.resourcemanager.bind-host 0.0.0.0 Default value is 0.0.0.0, when this is set the service will bind on all interfaces. I think these two options (blank, "0.0.0.0" sans quotes) should be the two available values, with blank as the default. + + yarn.nodemanager.bind-host 0.0.0.0 Default value is 0.0.0.0, when this is set the service will bind on all interfaces. I think these two options (blank, "0.0.0.0" sans quotes) should be the two available values, with blank as the default. + + yarn.timeline-service.bind-host 0.0.0.0 Default value is 0.0.0.0, when this is set the service will bind on all interfaces. I think these two options (blank, "0.0.0.0" sans quotes) should be the two available values, with blank as the default. + + yarn.node-labels.fs-store.root-dir /system/yarn/node-labels - + + + yarn.scheduler.minimum-allocation-vcores 1 - +

YARN Container Minimum vCores

int @@ -351,11 +452,13 @@ yarn.nodemanager.resource.cpu-vcores

+ + yarn.scheduler.maximum-allocation-vcores 8 - +

YARN Container Maximum vCores

int @@ -368,5 +471,7 @@ yarn.nodemanager.resource.cpu-vcores

+ + http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/configuration/accumulo-env.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/configuration/accumulo-env.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/configuration/accumulo-env.xml index b4259c0..243acfa 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/configuration/accumulo-env.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/configuration/accumulo-env.xml @@ -19,7 +19,6 @@ * limitations under the License. */ --> - accumulo_instance_name @@ -30,6 +29,7 @@ true false + + - http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/configuration/accumulo-log4j.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/configuration/accumulo-log4j.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/configuration/accumulo-log4j.xml index 9d34e3d..a86ed01 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/configuration/accumulo-log4j.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/configuration/accumulo-log4j.xml @@ -19,46 +19,50 @@ * limitations under the License. */ --> - - audit_log_level OFF Log level for audit logging + + - monitor_forwarding_log_level WARN Log level for logging forwarded to the Accumulo Monitor + + - debug_log_size 512M Size of each debug rolling log file + + - debug_num_logs 10 Number of rolling debug log files to keep + + - info_log_size 512M Size of each info rolling log file + + - info_num_logs 10 Number of rolling info log files to keep + + - content accumulo-log4j template @@ -112,6 +116,7 @@ log4j.appender.A1.layout=org.apache.log4j.PatternLayout content false + + - http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/configuration/accumulo-site.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/configuration/accumulo-site.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/configuration/accumulo-site.xml index cf176d4..52f5044 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/configuration/accumulo-site.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/configuration/accumulo-site.xml @@ -16,11 +16,9 @@ See the License for the specific language governing permissions and limitations under the License. --> - - general.classpaths @@ -46,5 +44,7 @@ $HADOOP_CONF_DIR, content + + http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/FALCON/configuration/falcon-env.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/FALCON/configuration/falcon-env.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/FALCON/configuration/falcon-env.xml index 8875286..9a7e6d2 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/FALCON/configuration/falcon-env.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/FALCON/configuration/falcon-env.xml @@ -1,3 +1,4 @@ + - *.shared.libs activemq-core,ant,geronimo-j2ee-management,jms,json-simple,oozie-client,spring-jms,commons-lang3,commons-el - + + + - http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/hbase-env.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/hbase-env.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/hbase-env.xml index c1cfc1e..97bccaf 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/hbase-env.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/hbase-env.xml @@ -19,7 +19,6 @@ * limitations under the License. */ --> - @@ -106,6 +105,7 @@ export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS {% if hbase_max_direct_ content + + - http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/hbase-site.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/hbase-site.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/hbase-site.xml index 45d7f8c..c395914 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/hbase-site.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/hbase-site.xml @@ -29,30 +29,42 @@ false int + + hbase.master.info.port 16010 The port for the HBase Master web UI. + + hbase.regionserver.port 16020 The port the HBase RegionServer binds to. + + hbase.regionserver.info.port 16030 The port for the HBase RegionServer web UI. + + hbase.regionserver.global.memstore.upperLimit 0.4 true + + hbase.regionserver.global.memstore.lowerLimit 0.38 true + + http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-audit.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-audit.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-audit.xml index f79f0ff..598b325 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-audit.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-audit.xml @@ -19,13 +19,13 @@ */ --> - xasecure.audit.is.enabled true Is Audit enabled? + + - xasecure.audit.destination.db false @@ -40,20 +40,23 @@ xasecure.audit.destination.db + + - xasecure.audit.destination.db.jdbc.url {{audit_jdbc_url}} Audit DB JDBC URL + + - xasecure.audit.destination.db.user {{xa_audit_db_user}} Audit DB JDBC User + + - xasecure.audit.destination.db.password crypted @@ -62,26 +65,30 @@ password + + - xasecure.audit.destination.db.jdbc.driver {{jdbc_driver}} Audit DB JDBC Driver + + - xasecure.audit.credential.provider.file jceks://file{{credential_file}} Credential file store + + - xasecure.audit.destination.db.batch.filespool.dir /var/log/hbase/audit/db/spool /var/log/hbase/audit/db/spool + + - xasecure.audit.destination.hdfs true @@ -96,8 +103,9 @@ xasecure.audit.destination.hdfs + + - xasecure.audit.destination.hdfs.dir hdfs://NAMENODE_HOSTNAME:8020/ranger/audit @@ -108,14 +116,16 @@ xasecure.audit.destination.hdfs.dir + + - xasecure.audit.destination.hdfs.batch.filespool.dir /var/log/hbase/audit/hdfs/spool /var/log/hbase/audit/hdfs/spool + + - xasecure.audit.destination.solr false @@ -130,11 +140,12 @@ xasecure.audit.destination.solr + + - xasecure.audit.destination.solr.urls - + Solr URL

true @@ -145,8 +156,9 @@ ranger.audit.solr.urls

+ + - xasecure.audit.destination.solr.zookeepers NONE @@ -157,14 +169,16 @@ ranger.audit.solr.zookeepers + + - xasecure.audit.destination.solr.batch.filespool.dir /var/log/hbase/audit/solr/spool /var/log/hbase/audit/solr/spool + + - xasecure.audit.provider.summary.enabled true @@ -173,6 +187,7 @@ boolean + + - http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-plugin-properties.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-plugin-properties.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-plugin-properties.xml index bf87456..1b41ec0 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-plugin-properties.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-plugin-properties.xml @@ -19,135 +19,160 @@ */ --> - XAAUDIT.DB.IS_ENABLED true - - + + + XAAUDIT.HDFS.IS_ENABLED true + + - XAAUDIT.HDFS.DESTINATION_DIRECTORY true + + - XAAUDIT.HDFS.LOCAL_BUFFER_DIRECTORY true + + - XAAUDIT.HDFS.LOCAL_ARCHIVE_DIRECTORY true + + - XAAUDIT.HDFS.DESTINTATION_FILE true + + - XAAUDIT.HDFS.DESTINTATION_FLUSH_INTERVAL_SECONDS true + + - XAAUDIT.HDFS.DESTINTATION_ROLLOVER_INTERVAL_SECONDS true + + - XAAUDIT.HDFS.DESTINTATION_OPEN_RETRY_INTERVAL_SECONDS true + + - XAAUDIT.HDFS.LOCAL_BUFFER_FILE true + + - XAAUDIT.HDFS.LOCAL_BUFFER_FLUSH_INTERVAL_SECONDS true + + - XAAUDIT.HDFS.LOCAL_BUFFER_ROLLOVER_INTERVAL_SECONDS true + + - XAAUDIT.HDFS.LOCAL_ARCHIVE_MAX_FILE_COUNT true + + - SSL_KEYSTORE_FILE_PATH true + + - SSL_KEYSTORE_PASSWORD true + + - SSL_TRUSTSTORE_FILE_PATH true + + - SSL_TRUSTSTORE_PASSWORD true + + - UPDATE_XAPOLICIES_ON_GRANT_REVOKE true + + - POLICY_MGR_URL - true - - + true + + + SQL_CONNECTOR_JAR true - - + + + XAAUDIT.DB.FLAVOUR - true - - + true + + + XAAUDIT.DB.DATABASE_NAME - true - - + true + + + XAAUDIT.DB.USER_NAME - true - - + true + + + XAAUDIT.DB.PASSWORD - true + true + + - XAAUDIT.DB.HOSTNAME - true + true + + - REPOSITORY_NAME - true + true + + - - \ No newline at end of file + http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-policymgr-ssl.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-policymgr-ssl.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-policymgr-ssl.xml index 8f349df..2f84158 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-policymgr-ssl.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-policymgr-ssl.xml @@ -19,49 +19,54 @@ */ --> - - - xasecure.policymgr.clientssl.keystore - /usr/hdp/current/hbase-client/conf/ranger-plugin-keystore.jks - Java Keystore files - - - - xasecure.policymgr.clientssl.keystore.password - myKeyFilePassword - PASSWORD - password for keystore - - password - - - - - xasecure.policymgr.clientssl.truststore - /usr/hdp/current/hbase-client/conf/ranger-plugin-truststore.jks - java truststore file - - - - xasecure.policymgr.clientssl.truststore.password - changeit - PASSWORD - java truststore password - - password - - - - - xasecure.policymgr.clientssl.keystore.credential.file - jceks://file{{credential_file}} - java keystore credential file - - - - xasecure.policymgr.clientssl.truststore.credential.file - jceks://file{{credential_file}} - java truststore credential file - - - \ No newline at end of file + + xasecure.policymgr.clientssl.keystore + /usr/hdp/current/hbase-client/conf/ranger-plugin-keystore.jks + Java Keystore files + + + + + xasecure.policymgr.clientssl.keystore.password + myKeyFilePassword + PASSWORD + password for keystore + + password + + + + + + xasecure.policymgr.clientssl.truststore + /usr/hdp/current/hbase-client/conf/ranger-plugin-truststore.jks + java truststore file + + + + + xasecure.policymgr.clientssl.truststore.password + changeit + PASSWORD + java truststore password + + password + + + + + + xasecure.policymgr.clientssl.keystore.credential.file + jceks://file{{credential_file}} + java keystore credential file + + + + + xasecure.policymgr.clientssl.truststore.credential.file + jceks://file{{credential_file}} + java truststore credential file + + + + http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-security.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-security.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-security.xml index de739b7..678be3a 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-security.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-security.xml @@ -23,38 +23,44 @@ ranger.plugin.hbase.service.name {{repo_name}} Name of the Ranger service containing HBase policies + + - ranger.plugin.hbase.policy.source.impl org.apache.ranger.admin.client.RangerAdminRESTClient Class to retrieve policies from the source + + - ranger.plugin.hbase.policy.rest.url {{policymgr_mgr_url}} URL to Ranger Admin + + - ranger.plugin.hbase.policy.rest.ssl.config.file /etc/hbase/conf/ranger-policymgr-ssl.xml Path to the file containing SSL details to contact Ranger Admin + + - ranger.plugin.hbase.policy.pollIntervalMs 30000 How often to poll for changes in policies? + + - ranger.plugin.hbase.policy.cache.dir /etc/ranger/{{repo_name}}/policycache Directory where Ranger policies are cached after successful retrieval from the source + + - xasecure.hbase.update.xapolicies.on.grant.revoke true @@ -63,6 +69,7 @@ boolean + + - - \ No newline at end of file + http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml index 2118fff..03c9f59 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml @@ -19,7 +19,6 @@ * limitations under the License. */ --> - @@ -131,7 +130,7 @@ export HADOOP_IDENT_STRING=$USER # Add database libraries JAVA_JDBC_LIBS="" if [ -d "/usr/share/java" ]; then - for jarFile in `ls /usr/share/java | grep -E "(mysql|ojdbc|postgresql|sqljdbc)" 2>/dev/null` + for jarFile in `ls /usr/share/java | grep -E "(mysql|ojdbc|postgresql|sqljdbc)" 2>/dev/null` do JAVA_JDBC_LIBS=${JAVA_JDBC_LIBS}:$jarFile done @@ -151,7 +150,7 @@ export HADOOP_OPTS="-Dhdp.version=$HDP_VERSION $HADOOP_OPTS" {% if is_datanode_max_locked_memory_set %} # Fix temporary bug, when ulimit from conf files is not picked up, without full relogin. # Makes sense to fix only when runing DN as root -if [ "$command" == "datanode" ] && [ "$EUID" -eq 0 ] && [ -n "$HADOOP_SECURE_DN_USER" ]; then +if [ "$command" == "datanode" ] && [ "$EUID" -eq 0 ] && [ -n "$HADOOP_SECURE_DN_USER" ]; then ulimit -l {{datanode_max_locked_memory}} fi {% endif %} @@ -159,6 +158,8 @@ fi content + + nfsgateway_heapsize @@ -169,6 +170,7 @@ fi int MB + + - http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hdfs-site.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hdfs-site.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hdfs-site.xml index 3a94b1c..86be42a 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hdfs-site.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hdfs-site.xml @@ -1,6 +1,5 @@ - - - - nfs.file.dump.dir /tmp/.hdfs-nfs @@ -33,16 +29,18 @@ One needs to make sure the directory has enough space. - directory + directory + + - nfs.exports.allowed.hosts * rw Allowed hosts + + - dfs.encrypt.data.transfer.cipher.suites AES/CTR/NoPadding @@ -52,8 +50,9 @@ If not defined, then only the algorithm specified in dfs.encrypt.data.transfer.algorithm is used. By default, the property is not defined. + + - dfs.namenode.inode.attributes.provider.class Enable ranger hdfs plugin @@ -66,6 +65,7 @@ false + + - http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-audit.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-audit.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-audit.xml index 703b078..ae11f34 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-audit.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-audit.xml @@ -19,13 +19,13 @@ */ --> - xasecure.audit.is.enabled true Is Audit enabled? + + - xasecure.audit.destination.db false @@ -40,20 +40,23 @@ xasecure.audit.destination.db + + - xasecure.audit.destination.db.jdbc.url {{audit_jdbc_url}} Audit DB JDBC URL + + - xasecure.audit.destination.db.user {{xa_audit_db_user}} Audit DB JDBC User + + - xasecure.audit.destination.db.password crypted @@ -62,26 +65,30 @@ password + + - xasecure.audit.destination.db.jdbc.driver {{jdbc_driver}} Audit DB JDBC Driver + + - xasecure.audit.credential.provider.file jceks://file{{credential_file}} Credential file store + + - xasecure.audit.destination.db.batch.filespool.dir /var/log/hadoop/hdfs/audit/db/spool /var/log/hadoop/hdfs/audit/db/spool + + - xasecure.audit.destination.hdfs true @@ -96,8 +103,9 @@ xasecure.audit.destination.hdfs + + - xasecure.audit.destination.hdfs.dir hdfs://NAMENODE_HOSTNAME:8020/ranger/audit @@ -108,14 +116,16 @@ xasecure.audit.destination.hdfs.dir + + - xasecure.audit.destination.hdfs.batch.filespool.dir /var/log/hadoop/hdfs/audit/hdfs/spool /var/log/hadoop/hdfs/audit/hdfs/spool + + - xasecure.audit.destination.solr false @@ -130,11 +140,12 @@ xasecure.audit.destination.solr + + - xasecure.audit.destination.solr.urls - + Solr URL

true @@ -145,8 +156,9 @@ ranger.audit.solr.urls

+ + - xasecure.audit.destination.solr.zookeepers NONE @@ -157,14 +169,16 @@ ranger.audit.solr.zookeepers + + - xasecure.audit.destination.solr.batch.filespool.dir /var/log/hadoop/hdfs/audit/solr/spool /var/log/hadoop/hdfs/audit/solr/spool + + - xasecure.audit.provider.summary.enabled false @@ -173,6 +187,7 @@ boolean + + - http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-plugin-properties.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-plugin-properties.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-plugin-properties.xml index 47af990..d8c0970 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-plugin-properties.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-plugin-properties.xml @@ -19,130 +19,154 @@ */ --> - XAAUDIT.DB.IS_ENABLED true + + - XAAUDIT.HDFS.IS_ENABLED true + + - XAAUDIT.HDFS.DESTINATION_DIRECTORY true + + - XAAUDIT.HDFS.LOCAL_BUFFER_DIRECTORY true + + - XAAUDIT.HDFS.LOCAL_ARCHIVE_DIRECTORY true + + - XAAUDIT.HDFS.DESTINTATION_FILE true + + - XAAUDIT.HDFS.DESTINTATION_FLUSH_INTERVAL_SECONDS true + + - XAAUDIT.HDFS.DESTINTATION_ROLLOVER_INTERVAL_SECONDS true + + - XAAUDIT.HDFS.DESTINTATION_OPEN_RETRY_INTERVAL_SECONDS true + + - XAAUDIT.HDFS.LOCAL_BUFFER_FILE true + + - XAAUDIT.HDFS.LOCAL_BUFFER_FLUSH_INTERVAL_SECONDS true + + - XAAUDIT.HDFS.LOCAL_BUFFER_ROLLOVER_INTERVAL_SECONDS true + + - XAAUDIT.HDFS.LOCAL_ARCHIVE_MAX_FILE_COUNT true + + - SSL_KEYSTORE_FILE_PATH true + + - SSL_KEYSTORE_PASSWORD true + + - SSL_TRUSTSTORE_FILE_PATH true + + - SSL_TRUSTSTORE_PASSWORD true - - + + + POLICY_MGR_URL true + + - SQL_CONNECTOR_JAR true + + - XAAUDIT.DB.FLAVOUR true + + - XAAUDIT.DB.DATABASE_NAME true + + - XAAUDIT.DB.USER_NAME true + + - XAAUDIT.DB.PASSWORD true + + - XAAUDIT.DB.HOSTNAME true + + - REPOSITORY_NAME true - - - \ No newline at end of file + + + + http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-policymgr-ssl.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-policymgr-ssl.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-policymgr-ssl.xml index f526de6..d74fd4d 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-policymgr-ssl.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-policymgr-ssl.xml @@ -19,49 +19,54 @@ */ --> - - - xasecure.policymgr.clientssl.keystore - /usr/hdp/current/hadoop-client/conf/ranger-plugin-keystore.jks - Java Keystore files - - - - xasecure.policymgr.clientssl.keystore.password - myKeyFilePassword - PASSWORD - password for keystore - - password - - - - - xasecure.policymgr.clientssl.truststore - /usr/hdp/current/hadoop-client/conf/ranger-plugin-truststore.jks - java truststore file - - - - xasecure.policymgr.clientssl.truststore.password - changeit - PASSWORD - java truststore password - - password - - - - - xasecure.policymgr.clientssl.keystore.credential.file - jceks://file{{credential_file}} - java keystore credential file - - - - xasecure.policymgr.clientssl.truststore.credential.file - jceks://file{{credential_file}} - java truststore credential file - - - \ No newline at end of file + + xasecure.policymgr.clientssl.keystore + /usr/hdp/current/hadoop-client/conf/ranger-plugin-keystore.jks + Java Keystore files + + + + + xasecure.policymgr.clientssl.keystore.password + myKeyFilePassword + PASSWORD + password for keystore + + password + + + + + + xasecure.policymgr.clientssl.truststore + /usr/hdp/current/hadoop-client/conf/ranger-plugin-truststore.jks + java truststore file + + + + + xasecure.policymgr.clientssl.truststore.password + changeit + PASSWORD + java truststore password + + password + + + + + + xasecure.policymgr.clientssl.keystore.credential.file + jceks://file{{credential_file}} + java keystore credential file + + + + + xasecure.policymgr.clientssl.truststore.credential.file + jceks://file{{credential_file}} + java truststore credential file + + + + http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-security.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-security.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-security.xml index 1bea198..2d35669 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-security.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-security.xml @@ -23,42 +23,49 @@ ranger.plugin.hdfs.service.name {{repo_name}} Name of the Ranger service containing Hdfs policies + + - ranger.plugin.hdfs.policy.source.impl org.apache.ranger.admin.client.RangerAdminRESTClient Class to retrieve policies from the source + + - ranger.plugin.hdfs.policy.rest.url {{policymgr_mgr_url}} URL to Ranger Admin + + - ranger.plugin.hdfs.policy.rest.ssl.config.file /etc/hadoop/conf/ranger-policymgr-ssl.xml Path to the file containing SSL details to contact Ranger Admin + + - ranger.plugin.hdfs.policy.pollIntervalMs 30000 How often to poll for changes in policies? + + - ranger.plugin.hdfs.policy.cache.dir /etc/ranger/{{repo_name}}/policycache Directory where Ranger policies are cached after successful retrieval from the source + + - xasecure.add-hadoop-authorization true Enable/Disable the default hadoop authorization (based on rwxrwxrwx permission on the resource) if Ranger Authorization fails. - - - \ No newline at end of file + + + + http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/hive-env.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/hive-env.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/hive-env.xml index f41ce67..b21dcc1 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/hive-env.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/hive-env.xml @@ -19,9 +19,7 @@ * limitations under the License. */ --> - - content @@ -80,6 +78,7 @@ export JAVA_LIBRARY_PATH="$JAVA_LIBRARY_PATH:{{jdbc_libs_dir}}" content + + - http://git-wip-us.apache.org/repos/asf/ambari/blob/6919aa50/ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/hive-site.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/hive-site.xml b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/hive-site.xml index de2813c..1f2178e 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/hive-site.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/hive-site.xml @@ -16,9 +16,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> - - hive.default.fileformat.managed TextFile @@ -27,8 +25,9 @@ limitations under the License. External tables will be created with default file format. Leaving this null will result in using the default file format for all tables. + + - datanucleus.rdbms.datastoreAdapterClassName Datanucleus Class, This property used only when hive db is SQL Anywhere @@ -38,8 +37,9 @@ limitations under the License. hive_database + + - atlas.hook.hive.minThreads DONT_ADD_ON_UPGRADE @@ -47,8 +47,9 @@ limitations under the License. Minimum number of threads maintained by Atlas hook. + + -

DONT_ADD_ON_UPGRADE

atlas.hook.hive.maxThreads @@ -56,6 +57,7 @@ limitations under the License. Maximum number of threads used by Atlas hook. + + -