Return-Path: X-Original-To: apmail-ambari-commits-archive@www.apache.org Delivered-To: apmail-ambari-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id F2D9418A1D for ; Fri, 5 Jun 2015 17:30:06 +0000 (UTC) Received: (qmail 24002 invoked by uid 500); 5 Jun 2015 17:30:06 -0000 Delivered-To: apmail-ambari-commits-archive@ambari.apache.org Received: (qmail 23972 invoked by uid 500); 5 Jun 2015 17:30:06 -0000 Mailing-List: contact commits-help@ambari.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: ambari-dev@ambari.apache.org Delivered-To: mailing list commits@ambari.apache.org Received: (qmail 23963 invoked by uid 99); 5 Jun 2015 17:30:06 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Jun 2015 17:30:06 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id B903EE04E6; Fri, 5 Jun 2015 17:30:06 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: dsen@apache.org To: commits@ambari.apache.org Message-Id: <500130e96c29459c932a7918c8b970a1@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: ambari git commit: AMBARI-11727 Perf: Set appropriate ulimit values for Metrics Hbase Master and Collector processes (dsen) Date: Fri, 5 Jun 2015 17:30:06 +0000 (UTC) Repository: ambari Updated Branches: refs/heads/branch-2.1 cdd1a68fb -> 3004eee05 AMBARI-11727 Perf: Set appropriate ulimit values for Metrics Hbase Master and Collector processes (dsen) Project: http://git-wip-us.apache.org/repos/asf/ambari/repo Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/3004eee0 Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/3004eee0 Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/3004eee0 Branch: refs/heads/branch-2.1 Commit: 3004eee0535c923b9f46d5352c4e177984109bfd Parents: cdd1a68 Author: Dmytro Sen Authored: Fri Jun 5 20:25:06 2015 +0300 Committer: Dmytro Sen Committed: Fri Jun 5 20:26:12 2015 +0300 ---------------------------------------------------------------------- .../0.1.0/configuration/ams-hbase-env.xml | 7 ++++ .../AMBARI_METRICS/0.1.0/package/scripts/ams.py | 15 +++++++++ .../0.1.0/package/scripts/params.py | 2 ++ .../0.1.0/package/scripts/params_linux.py | 2 ++ .../0.1.0/package/templates/ams.conf.j2 | 35 ++++++++++++++++++++ 5 files changed, 61 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/ambari/blob/3004eee0/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml index 1734ac4..6325a50 100644 --- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml +++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml @@ -69,6 +69,13 @@ sum of master and regionserver heap sizes. + + max_open_files_limit + 32768 + + The maximum number of open file descriptors by process + + http://git-wip-us.apache.org/repos/asf/ambari/blob/3004eee0/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py index fc18042..b08e705 100644 --- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py +++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py @@ -245,6 +245,21 @@ def ams(name=None): mode=0755 ) + # On some OS this folder could be not exists, so we will create it before pushing there files + Directory(params.limits_conf_dir, + recursive=True, + owner='root', + group='root' + ) + + # Setting up security limits + File(os.path.join(params.limits_conf_dir, 'ams.conf'), + owner='root', + group='root', + mode=0644, + content=Template("ams.conf.j2") + ) + # Phoenix spool file dir if not /tmp if not os.path.exists(params.phoenix_client_spool_dir): Directory(params.phoenix_client_spool_dir, http://git-wip-us.apache.org/repos/asf/ambari/blob/3004eee0/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py index 47fd2dd..4c78f77 100644 --- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py +++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py @@ -110,6 +110,8 @@ if not is_hbase_distributed: else: hbase_heapsize = master_heapsize +max_open_files_limit = default("/configurations/ams-hbase-env/max_open_files_limit", "32768") + zookeeper_quorum_hosts = ','.join(ams_collector_hosts) if is_hbase_distributed else 'localhost' ams_checkpoint_dir = config['configurations']['ams-site']['timeline.metrics.aggregator.checkpoint.dir'] http://git-wip-us.apache.org/repos/asf/ambari/blob/3004eee0/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params_linux.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params_linux.py b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params_linux.py index e0085ba..6443fe8 100644 --- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params_linux.py +++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params_linux.py @@ -44,3 +44,5 @@ hbase_cmd = "/usr/lib/ams-hbase/bin/hbase" hadoop_conf_dir = conf_select.get_hadoop_conf_dir() hbase_conf_dir = "/etc/ams-hbase/conf" + +limits_conf_dir = "/etc/security/limits.d" http://git-wip-us.apache.org/repos/asf/ambari/blob/3004eee0/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/ams.conf.j2 ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/ams.conf.j2 b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/ams.conf.j2 new file mode 100644 index 0000000..c5fbc9b --- /dev/null +++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/ams.conf.j2 @@ -0,0 +1,35 @@ +{# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} + +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +{{ams_user}} - nofile {{max_open_files_limit}} +{{ams_user}} - nproc 65536