Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id ECB792004A0 for ; Wed, 16 Aug 2017 11:02:36 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id E9CB41685A0; Wed, 16 Aug 2017 09:02:36 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id E261716859F for ; Wed, 16 Aug 2017 11:02:35 +0200 (CEST) Received: (qmail 2169 invoked by uid 500); 16 Aug 2017 09:02:34 -0000 Mailing-List: contact commits-help@ambari.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: ambari-dev@ambari.apache.org Delivered-To: mailing list commits@ambari.apache.org Received: (qmail 2160 invoked by uid 99); 16 Aug 2017 09:02:34 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 Aug 2017 09:02:34 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id A3D4AE0262; Wed, 16 Aug 2017 09:02:33 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: aonishuk@apache.org To: commits@ambari.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: ambari git commit: AMBARI-21717. Additional fixes to stacks config paths (aonishuk) Date: Wed, 16 Aug 2017 09:02:33 +0000 (UTC) archived-at: Wed, 16 Aug 2017 09:02:37 -0000 Repository: ambari Updated Branches: refs/heads/branch-3.0-perf 57b79f937 -> 9b9402e93 AMBARI-21717. Additional fixes to stacks config paths (aonishuk) Project: http://git-wip-us.apache.org/repos/asf/ambari/repo Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/9b9402e9 Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/9b9402e9 Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/9b9402e9 Branch: refs/heads/branch-3.0-perf Commit: 9b9402e9305a879f83a0d0182c70b73e166a68e8 Parents: 57b79f9 Author: Andrew Onishuk Authored: Wed Aug 16 12:00:00 2017 +0300 Committer: Andrew Onishuk Committed: Wed Aug 16 12:00:00 2017 +0300 ---------------------------------------------------------------------- .../0.12.0.2.0/package/scripts/params_linux.py | 2 +- .../HIVE/2.1.0.3.0/package/scripts/params_linux.py | 2 +- .../KERBEROS/1.10.3-10/package/scripts/params.py | 2 +- .../PXF/3.0.0/package/scripts/params.py | 4 ++-- .../RANGER_KMS/0.5.0.2.3/package/scripts/params.py | 2 +- .../TEZ/0.4.0.2.1/package/scripts/tez_client.py | 2 +- .../TEZ/0.9.0.3.0/package/scripts/tez_client.py | 2 +- .../2.0.6/hooks/before-INSTALL/scripts/params.py | 2 +- .../HDP/2.0.6/hooks/before-START/scripts/params.py | 17 ++++------------- .../before-START/scripts/shared_initialization.py | 2 +- .../PERF/1.0/hooks/before-ANY/scripts/params.py | 4 ++-- 11 files changed, 16 insertions(+), 25 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/ambari/blob/9b9402e9/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py b/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py index 1ed564e..49b67f6 100644 --- a/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py +++ b/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py @@ -252,7 +252,7 @@ hive_user = config['configurations']['hive-env']['hive_user'] # is it a restart command is_restart_command = False if 'roleCommand' in config and 'CUSTOM_COMMAND' == config['roleCommand']: - if 'custom_command' in config['hostLevelParams'] and 'RESTART' == config['hostLevelParams']['custom_command']: + if 'custom_command' in config['commandParams'] and 'RESTART' == config['commandParams']['custom_command']: is_restart_command = True #JDBC driver jar name http://git-wip-us.apache.org/repos/asf/ambari/blob/9b9402e9/ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/params_linux.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/params_linux.py b/ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/params_linux.py index 1ed564e..49b67f6 100644 --- a/ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/params_linux.py +++ b/ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/params_linux.py @@ -252,7 +252,7 @@ hive_user = config['configurations']['hive-env']['hive_user'] # is it a restart command is_restart_command = False if 'roleCommand' in config and 'CUSTOM_COMMAND' == config['roleCommand']: - if 'custom_command' in config['hostLevelParams'] and 'RESTART' == config['hostLevelParams']['custom_command']: + if 'custom_command' in config['commandParams'] and 'RESTART' == config['commandParams']['custom_command']: is_restart_command = True #JDBC driver jar name http://git-wip-us.apache.org/repos/asf/ambari/blob/9b9402e9/ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/package/scripts/params.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/package/scripts/params.py b/ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/package/scripts/params.py index 3bde62d..4809e6e 100644 --- a/ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/package/scripts/params.py +++ b/ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/package/scripts/params.py @@ -66,7 +66,7 @@ smoke_user = 'ambari-qa' manage_identities = 'true' artifact_dir = format("{tmp_dir}/AMBARI-artifacts/") -jce_policy_zip = default("/hostLevelParams/jce_name", None) # None when jdk is already installed by user +jce_policy_zip = default("/ambariLevelParams/jce_name", None) # None when jdk is already installed by user jce_location = config['ambariLevelParams']['jdk_location'] jdk_name = default("/ambariLevelParams/jdk_name", None) java_home = config['ambariLevelParams']['java_home'] http://git-wip-us.apache.org/repos/asf/ambari/blob/9b9402e9/ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py b/ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py index cfe7461..64d7c5a 100644 --- a/ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py +++ b/ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py @@ -28,7 +28,7 @@ from resource_management.libraries.functions.get_not_managed_resources import ge config = Script.get_config() pxf_service_name = pxf_constants.pxf_service_name -stack_name = str(config["hostLevelParams"]["stack_name"]) +stack_name = str(config["clusterLevelParams"]["stack_name"]) # Users and Groups pxf_user = pxf_constants.pxf_user @@ -46,7 +46,7 @@ pxf_instance_dir = "/var/pxf" exec_tmp_dir = Script.get_tmp_dir() # Java home path -java_home = config["hostLevelParams"]["java_home"] if "java_home" in config["hostLevelParams"] else None +java_home = config["ambariLevelParams"]["java_home"] if "java_home" in config["ambariLevelParams"] else None # Timeouts default_exec_timeout = 600 http://git-wip-us.apache.org/repos/asf/ambari/blob/9b9402e9/ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py b/ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py index 65ba704..4152dfb 100755 --- a/ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py +++ b/ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py @@ -241,7 +241,7 @@ check_db_connection_jar = format("/usr/lib/ambari-agent/{check_db_connection_jar ranger_kms_jdbc_connection_url = config['configurations']['dbks-site']['ranger.ks.jpa.jdbc.url'] ranger_kms_jdbc_driver = config['configurations']['dbks-site']['ranger.ks.jpa.jdbc.driver'] -jce_name = default("/hostLevelParams/jce_name", None) +jce_name = default("/ambariLevelParams/jce_name", None) jce_source_dir = format('{tmp_dir}/jce_dir') #kms hsm support http://git-wip-us.apache.org/repos/asf/ambari/blob/9b9402e9/ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py b/ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py index da2c71a..c054b65 100644 --- a/ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py +++ b/ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py @@ -117,7 +117,7 @@ class TezClientWindows(TezClient): config = Script.get_config() file_url = urlparse.urljoin(config['ambariLevelParams']['jdk_location'], hadoop_lzo_file) - hadoop_lzo_dl_path = os.path.join(config["hostLevelParams"]["agentCacheDir"], hadoop_lzo_file) + hadoop_lzo_dl_path = os.path.join(config["agentLevelParams"]["agentCacheDir"], hadoop_lzo_file) download_file(file_url, hadoop_lzo_dl_path) #This is for protection against configuration changes. It will infect every new destination with the lzo jar, # but since the classpath points to the jar directly we're getting away with it. http://git-wip-us.apache.org/repos/asf/ambari/blob/9b9402e9/ambari-server/src/main/resources/common-services/TEZ/0.9.0.3.0/package/scripts/tez_client.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/common-services/TEZ/0.9.0.3.0/package/scripts/tez_client.py b/ambari-server/src/main/resources/common-services/TEZ/0.9.0.3.0/package/scripts/tez_client.py index da2c71a..c054b65 100644 --- a/ambari-server/src/main/resources/common-services/TEZ/0.9.0.3.0/package/scripts/tez_client.py +++ b/ambari-server/src/main/resources/common-services/TEZ/0.9.0.3.0/package/scripts/tez_client.py @@ -117,7 +117,7 @@ class TezClientWindows(TezClient): config = Script.get_config() file_url = urlparse.urljoin(config['ambariLevelParams']['jdk_location'], hadoop_lzo_file) - hadoop_lzo_dl_path = os.path.join(config["hostLevelParams"]["agentCacheDir"], hadoop_lzo_file) + hadoop_lzo_dl_path = os.path.join(config["agentLevelParams"]["agentCacheDir"], hadoop_lzo_file) download_file(file_url, hadoop_lzo_dl_path) #This is for protection against configuration changes. It will infect every new destination with the lzo jar, # but since the classpath points to the jar directly we're getting away with it. http://git-wip-us.apache.org/repos/asf/ambari/blob/9b9402e9/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/params.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/params.py b/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/params.py index a72f3da..a2b119d 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/params.py +++ b/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/params.py @@ -99,7 +99,7 @@ security_enabled = config['configurations']['cluster-env']['security_enabled'] java_home = config['ambariLevelParams']['java_home'] artifact_dir = format("{tmp_dir}/AMBARI-artifacts/") jdk_name = default("/ambariLevelParams/jdk_name", None) # None when jdk is already installed by user -jce_policy_zip = default("/hostLevelParams/jce_name", None) # None when jdk is already installed by user +jce_policy_zip = default("/ambariLevelParams/jce_name", None) # None when jdk is already installed by user jce_location = config['ambariLevelParams']['jdk_location'] jdk_location = config['ambariLevelParams']['jdk_location'] ignore_groupsusers_create = default("/configurations/cluster-env/ignore_groupsusers_create", False) http://git-wip-us.apache.org/repos/asf/ambari/blob/9b9402e9/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py b/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py index be16b63..1f13e03 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py +++ b/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py @@ -78,15 +78,15 @@ current_service = config['serviceName'] #security params security_enabled = config['configurations']['cluster-env']['security_enabled'] -ambari_server_resources_url = default("/hostLevelParams/jdk_location", None) +ambari_server_resources_url = default("/ambariLevelParams/jdk_location", None) if ambari_server_resources_url is not None and ambari_server_resources_url.endswith('/'): ambari_server_resources_url = ambari_server_resources_url[:-1] # Unlimited key JCE policy params -jce_policy_zip = default("/hostLevelParams/jce_name", None) # None when jdk is already installed by user -unlimited_key_jce_required = default("/hostLevelParams/unlimited_key_jce_required", False) +jce_policy_zip = default("/ambariLevelParams/jce_name", None) # None when jdk is already installed by user +unlimited_key_jce_required = default("/componentLevelParams/unlimited_key_jce_required", False) jdk_name = default("/ambariLevelParams/jdk_name", None) -java_home = default("/hostLevelParams/java_home", None) +java_home = default("/ambariLevelParams/java_home", None) java_exec = "{0}/bin/java".format(java_home) if java_home is not None else "/bin/java" #users and groups @@ -179,18 +179,9 @@ hadoop_pid_dir_prefix = config['configurations']['hadoop-env']['hadoop_pid_dir_p hdfs_log_dir_prefix = config['configurations']['hadoop-env']['hdfs_log_dir_prefix'] hbase_tmp_dir = "/tmp/hbase-hbase" #db params -server_db_name = config['hostLevelParams']['db_name'] -db_driver_filename = config['hostLevelParams']['db_driver_filename'] -oracle_driver_url = config['hostLevelParams']['oracle_jdbc_url'] -mysql_driver_url = config['hostLevelParams']['mysql_jdbc_url'] oracle_driver_symlink_url = format("{ambari_server_resources_url}/oracle-jdbc-driver.jar") mysql_driver_symlink_url = format("{ambari_server_resources_url}/mysql-jdbc-driver.jar") -ambari_db_rca_url = config['hostLevelParams']['ambari_db_rca_url'][0] -ambari_db_rca_driver = config['hostLevelParams']['ambari_db_rca_driver'][0] -ambari_db_rca_username = config['hostLevelParams']['ambari_db_rca_username'][0] -ambari_db_rca_password = config['hostLevelParams']['ambari_db_rca_password'][0] - if has_namenode and 'rca_enabled' in config['configurations']['hadoop-env']: rca_enabled = config['configurations']['hadoop-env']['rca_enabled'] else: http://git-wip-us.apache.org/repos/asf/ambari/blob/9b9402e9/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py b/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py index e246fbd..84a85cd 100644 --- a/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py +++ b/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py @@ -206,7 +206,7 @@ def setup_unlimited_key_jce_policy(): * The cluster has not been previously prepared (sys preped) - cluster-env/sysprep_skip_setup_jce = False * Ambari is managing the host's JVM - /ambariLevelParams/jdk_name is set - * Either security is enabled OR a service requires it - /hostLevelParams/unlimited_key_jce_required = True + * Either security is enabled OR a service requires it - /componentLevelParams/unlimited_key_jce_required = True * The unlimited key JCE policy has not already been installed If the conditions are met, the following steps are taken to install the unlimited key JCE policy JARs http://git-wip-us.apache.org/repos/asf/ambari/blob/9b9402e9/ambari-server/src/main/resources/stacks/PERF/1.0/hooks/before-ANY/scripts/params.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/resources/stacks/PERF/1.0/hooks/before-ANY/scripts/params.py b/ambari-server/src/main/resources/stacks/PERF/1.0/hooks/before-ANY/scripts/params.py index 836c809..3a1fbe1 100644 --- a/ambari-server/src/main/resources/stacks/PERF/1.0/hooks/before-ANY/scripts/params.py +++ b/ambari-server/src/main/resources/stacks/PERF/1.0/hooks/before-ANY/scripts/params.py @@ -36,8 +36,8 @@ user_to_groups_dict = collections.defaultdict(lambda:[user_group]) jdk_name = default("/ambariLevelParams/jdk_name", None) java_home = config['ambariLevelParams']['java_home'] artifact_dir = format("{tmp_dir}/AMBARI-artifacts/") -jdk_location = config['hostLevelParams']['jdk_location'] -java_version = expect("/hostLevelParams/java_version", int) +jdk_location = config['ambariLevelParams']['jdk_location'] +java_version = expect("/ambariLevelParams/java_version", int) service_name = config["serviceName"] component_name = config["role"]