Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id A395C200A5B for ; Wed, 25 May 2016 08:53:30 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id A238F160A18; Wed, 25 May 2016 06:53:30 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id E991B160A17 for ; Wed, 25 May 2016 08:53:29 +0200 (CEST) Received: (qmail 66468 invoked by uid 500); 25 May 2016 06:53:28 -0000 Mailing-List: contact commits-help@ambari.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: ambari-dev@ambari.apache.org Delivered-To: mailing list commits@ambari.apache.org Received: (qmail 66459 invoked by uid 99); 25 May 2016 06:53:28 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 25 May 2016 06:53:28 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id C7DB0E0950; Wed, 25 May 2016 06:53:28 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: stoader@apache.org To: commits@ambari.apache.org Message-Id: <5be79529f0c3476e96b715239963b532@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: ambari git commit: AMBARI-16776. HiveServer interactive - incorrect default memory value. (Daniel Gergely via stoader) Date: Wed, 25 May 2016 06:53:28 +0000 (UTC) archived-at: Wed, 25 May 2016 06:53:30 -0000 Repository: ambari Updated Branches: refs/heads/trunk 71a0689e7 -> f19904109 AMBARI-16776. HiveServer interactive - incorrect default memory value. (Daniel Gergely via stoader) Project: http://git-wip-us.apache.org/repos/asf/ambari/repo Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/f1990410 Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/f1990410 Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/f1990410 Branch: refs/heads/trunk Commit: f199041091ef54b2d0deccd1a414b018a1420e1b Parents: 71a0689 Author: Daniel Gergely Authored: Wed May 25 08:51:03 2016 +0200 Committer: Toader, Sebastian Committed: Wed May 25 08:53:09 2016 +0200 ---------------------------------------------------------------------- .../HIVE/0.12.0.2.0/package/scripts/params_linux.py | 2 ++ .../2.5/services/HIVE/configuration/hive-interactive-site.xml | 2 +- .../main/resources/stacks/HDP/2.5/services/stack_advisor.py | 6 ++++-- 3 files changed, 7 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/ambari/blob/f1990410/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py b/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py index f285853..76a417d 100644 --- a/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py +++ b/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py @@ -378,6 +378,8 @@ jars_in_hive_lib = format("{hive_lib}/*.jar") start_hiveserver2_path = format("{tmp_dir}/start_hiveserver2_script") start_metastore_path = format("{tmp_dir}/start_metastore_script") +yarn_scheduler_allocation_minimum_mb = config['configurations']['yarn-site']['properties']['yarn.scheduler.minimum-allocation-mb'] + hadoop_heapsize = config['configurations']['hadoop-env']['hadoop_heapsize'] if 'role' in config and config['role'] in ["HIVE_SERVER", "HIVE_METASTORE"]: http://git-wip-us.apache.org/repos/asf/ambari/blob/f1990410/ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml b/ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml index db5f616..1ff8413 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml +++ b/ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml @@ -430,7 +430,7 @@ limitations under the License. hive.llap.daemon.yarn.container.mb YARN Memory per Daemon The total YARN memory per node to be used by Hive LLAP daemons. This includes memory for cache as well as for memory for query processing. - 341 + {{yarn_scheduler_allocation_minimum_mb}} int MB http://git-wip-us.apache.org/repos/asf/ambari/blob/f1990410/ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py b/ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py index 0512162..f07669b 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py +++ b/ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py @@ -227,6 +227,10 @@ class HDP25StackAdvisor(HDP24StackAdvisor): llap_queue_selected_in_current_call = None LLAP_MAX_CONCURRENCY = 32 # Allow a max of 32 concurrency. + # initial memory setting to make sure hive.llap.daemon.yarn.container.mb >= yarn.scheduler.minimum-allocation-mb + Logger.debug("Setting hive.llap.daemon.yarn.container.mb to yarn min container size as initial size (" + str(self.get_yarn_min_container_size(services)) + " MB).") + putHiveInteractiveSiteProperty('hive.llap.daemon.yarn.container.mb', long(self.get_yarn_min_container_size(services))) + try: if self.HIVE_INTERACTIVE_SITE in services['configurations'] and \ 'hive.llap.daemon.queue.name' in services['configurations'][self.HIVE_INTERACTIVE_SITE]['properties']: @@ -246,8 +250,6 @@ class HDP25StackAdvisor(HDP24StackAdvisor): Logger.debug("Selected YARN queue is '{0}'. Setting LLAP queue capacity slider visibility to True".format(llap_queue_name)) else: putHiveInteractiveEnvPropertyAttribute("llap_queue_capacity", "visible", "false") - Logger.debug("Setting hive.llap.daemon.yarn.container.mb to yarn min container size (" + str(self.get_yarn_min_container_size(services)) + " MB).") - putHiveInteractiveSiteProperty('hive.llap.daemon.yarn.container.mb', long(self.get_yarn_min_container_size(services))) Logger.debug("Queue selected for LLAP app is : '{0}'. Current YARN queues : {1}. " "Setting LLAP queue capacity slider visibility to False. " "Skipping updating values for LLAP related configs".format(llap_daemon_selected_queue_name, list(leafQueueNames)))