ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez" <afernan...@hortonworks.com>
Subject Re: Review Request 41432: AMBARI-14390. EU from HDP 2.1 to 2.3 added NULL properties
Date Wed, 16 Dec 2015 04:25:13 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41432/
-----------------------------------------------------------

(Updated Dec. 16, 2015, 4:25 a.m.)


Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya,
and Nate Cole.


Changes
-------

Unit tests passed


Bugs: AMBARI-14390
    https://issues.apache.org/jira/browse/AMBARI-14390


Repository: ambari


Description
-------

*Steps:*
With Ambari 2.2.0, setup HDP 2.1 cluster (non HA, unsecure) with following services: hbase,pig,slider,hive,oozie,sqoop,flume,falcon,storm
Did Express Upgrade from HDP 2.1 to 2.3.4 and let it finish
Observe the configs of various services

*Result:*
Observed that the values of several properties showed up as blank and required (mostly security
related). Here is the list:

HDFS:

* hdfs_principal_name (hadoop-env, value is null, remove if not kerberized)
* hdfs_user_keytab (hadoop-env, value is null, remove if not kerberized)
* dfs.namenode.inode.attributes.provider.class (hdfs-site, value is null, set to "org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer"
if Ranger is present, which it is not in HDP 2.1)

YARN:

* yarn.authorization-provider (yarn-site, value is null, set to "org.apache.ranger.authorization.yarn.authorizer.RangerYarnAuthorizer"
if Ranger is present, which it is not in HDP 2.1)

Tez:

* tez.tez-ui.history-url.base (also tracked by AMBARI-14389, tez-site, value is null, set
to {code} '{0}://{1}:{2}/#/main/views/TEZ/{3}/TEZ_CLUSTER_INSTANCE'.format(server_protocol,
server_host, server_port, latest_tez_jar_version) {code} by Stack Advisor)

HBase:

* hbase_principal_name (hbase-env, value is null, remove if not kerberized)
* hbase_user_keytab (hbase-env, value is null, remove if not kerberized)

ZK:

* zookeeper_keytab_path (zookeeper-env, value is null, remove if not kerberized)
* zookeeper_principal_name (zookeeper-env, value is null, remove if not kerberized)

Storm:

* nimbus_keytab  (storm-env, value is null, remove if not kerberized)
* nimbus_principal_name (storm-env, value is null, remove if not kerberized)
* storm_keytab (storm-env, value is null, remove if not kerberized)
* storm_principal_name (storm-env, value is null, remove if not kerberized)
* storm_ui_keytab (storm-env, value is null, remove if not kerberized)
* storm_ui_principal_name (storm-env, value is null, remove if not kerberized)
* nimbus.authorizer (storm-site, need help from Storm team on what to set this to since it
depends on whether kerberos is enabled and Ranger is present)


Diffs
-----

  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
1e59b58 
  ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java b799bc7

  ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml ef4b531

  ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/config-upgrade.xml ab384db 

Diff: https://reviews.apache.org/r/41432/diff/


Testing (updated)
-------

Installed Ambari 2.2.0 with HDP 2.1 and all of the services listed above, then performed Express
Upgrade to HDP 2.3 and verified through the configs page that no properties had null values.
This happened because of all of the kerberos-related properties in stack HDP 2.3 have null,
so the merging of configurations was adding the property.

mvn clean test

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:01:45.515s
[INFO] Finished at: Tue Dec 15 20:24:13 PST 2015
[INFO] Final Memory: 120M/4042M
[INFO] ------------------------------------------------------------------------


Thanks,

Alejandro Fernandez


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message