Mailing-List: contact dev-help@ambari.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ambari.apache.org
Content-Type: multipart/alternative;
 boundary="===============6224772940290622701=="
MIME-Version: 1.0
Subject: Re: Review Request 41432: AMBARI-14390. EU from HDP 2.1 to 2.3 added
 NULL properties
From: "Mahadev Konar" <mahadev@apache.org>
To: "Jayush Luniya" <jluniya@hortonworks.com>,
 "Dmitro Lisnichenko" <dlysnichenko@hortonworks.com>,
 "Dmytro Grinenko" <hapy.lestat@gmail.com>,
 "Nate Cole" <ncole@hortonworks.com>,
 "Jonathan Hurley" <jhurley@hortonworks.com>
Cc: "Mahadev Konar" <mahadev@apache.org>, "Ambari" <dev@ambari.apache.org>,
 "Alejandro Fernandez" <afernandez@hortonworks.com>
Date: Wed, 16 Dec 2015 04:47:48 -0000
Message-ID: <20151216044748.19749.34702@reviews.apache.org>
Auto-Submitted: auto-generated
Sender: "Mahadev Konar" <noreply@reviews.apache.org>
References: <20151216042513.31807.45095@reviews.apache.org>
In-Reply-To: <20151216042513.31807.45095@reviews.apache.org>
Reply-To: "Mahadev Konar" <mahadev@apache.org>

--===============6224772940290622701==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41432/#review110617
-----------------------------------------------------------

Ship it!


Ship It!

- Mahadev Konar


On Dec. 16, 2015, 4:25 a.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41432/
> -----------------------------------------------------------
> 
> (Updated Dec. 16, 2015, 4:25 a.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-14390
>     https://issues.apache.org/jira/browse/AMBARI-14390
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> *Steps:*
> With Ambari 2.2.0, setup HDP 2.1 cluster (non HA, unsecure) with following services: hbase,pig,slider,hive,oozie,sqoop,flume,falcon,storm
> Did Express Upgrade from HDP 2.1 to 2.3.4 and let it finish
> Observe the configs of various services
> 
> *Result:*
> Observed that the values of several properties showed up as blank and required (mostly security related). Here is the list:
> 
> HDFS:
> 
> * hdfs_principal_name (hadoop-env, value is null, remove if not kerberized)
> * hdfs_user_keytab (hadoop-env, value is null, remove if not kerberized)
> * dfs.namenode.inode.attributes.provider.class (hdfs-site, value is null, set to "org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer" if Ranger is present, which it is not in HDP 2.1)
> 
> YARN:
> 
> * yarn.authorization-provider (yarn-site, value is null, set to "org.apache.ranger.authorization.yarn.authorizer.RangerYarnAuthorizer" if Ranger is present, which it is not in HDP 2.1)
> 
> Tez:
> 
> * tez.tez-ui.history-url.base (also tracked by AMBARI-14389, tez-site, value is null, set to {code} '{0}://{1}:{2}/#/main/views/TEZ/{3}/TEZ_CLUSTER_INSTANCE'.format(server_protocol, server_host, server_port, latest_tez_jar_version) {code} by Stack Advisor)
> 
> HBase:
> 
> * hbase_principal_name (hbase-env, value is null, remove if not kerberized)
> * hbase_user_keytab (hbase-env, value is null, remove if not kerberized)
> 
> ZK:
> 
> * zookeeper_keytab_path (zookeeper-env, value is null, remove if not kerberized)
> * zookeeper_principal_name (zookeeper-env, value is null, remove if not kerberized)
> 
> Storm:
> 
> * nimbus_keytab  (storm-env, value is null, remove if not kerberized)
> * nimbus_principal_name (storm-env, value is null, remove if not kerberized)
> * storm_keytab (storm-env, value is null, remove if not kerberized)
> * storm_principal_name (storm-env, value is null, remove if not kerberized)
> * storm_ui_keytab (storm-env, value is null, remove if not kerberized)
> * storm_ui_principal_name (storm-env, value is null, remove if not kerberized)
> * nimbus.authorizer (storm-site, need help from Storm team on what to set this to since it depends on whether kerberos is enabled and Ranger is present)
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 1e59b58 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java b799bc7 
>   ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml ef4b531 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/config-upgrade.xml ab384db 
> 
> Diff: https://reviews.apache.org/r/41432/diff/
> 
> 
> Testing
> -------
> 
> Installed Ambari 2.2.0 with HDP 2.1 and all of the services listed above, then performed Express Upgrade to HDP 2.3 and verified through the configs page that no properties had null values.
> This happened because of all of the kerberos-related properties in stack HDP 2.3 have null, so the merging of configurations was adding the property.
> 
> mvn clean test
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 1:01:45.515s
> [INFO] Finished at: Tue Dec 15 20:24:13 PST 2015
> [INFO] Final Memory: 120M/4042M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>


--===============6224772940290622701==--