ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez" <afernan...@hortonworks.com>
Subject Review Request 38543: RU: Knox to use versioned data dir starting in HDP 2.3.2.0
Date Mon, 21 Sep 2015 05:16:17 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38543/
-----------------------------------------------------------

Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, Srimanth Gunturi, and
Sid Wagle.


Bugs: AMBARI-13164
    https://issues.apache.org/jira/browse/AMBARI-13164


Repository: ambari


Description
-------

Starting in HDP 2.3.2.0, Knox will version its data directory.

HDP 2.3.2.0 is meant to support Debian & Ubuntu.
A customer installing HDP 2.3.2.0 would receive the following for knox:
/usr/hdp/current/knox/data is a symlink to /var/lib/knox/data_$
{version}

Hence, all of the data directories in /var/lib/knox/ would be versioned. This is because the
Debian installer fails if it tries to create the /var/lib/knox/data folder and it already
exists (since it belongs to a previous version); further, you would get data loss if you tried
to uninstall the newer version, since it would wipe out the folder that was used by the previous
version.

So the problem doesn't arise on the initial install, but rather on installing an additional
version for Rolling Upgrade.

This means that any customer planning to upgrade Knox must first upgrade Ambari to 2.1.2.

There are 3 scenarios:

1. Ambari 2.1.2 with HDP 2.3.0.0:
Ambari must continue to use the non-versioned data directory.

2. Ambari 2.1.2 with HDP 2.3.2.0:
A brand new install will use the versioned data directory out-of-the-box.

3. Ambari 2.1.0 with HDP 2.3.0.0:
Technically, this allows registering bits for HDP 2.3.2.0 and performing an RU to it. However,
because HDP 2.3.2.0 uses a versioned data dir, it means that any customer with Knox that plans
to perform an RU must first upgrade Ambari to 2.1.2, which is equivalent to starting with
case #1.

Because RU allows the following paths, 
2.3.0.0 (non-versioned data dir) -> 2.3.2.0 (versioned data dir)
2.3.2.0 (versioned data dir) -> 2.3.3.- (versioned data dir, once it is available)

It means that it must always copy any contents from 
previous_data_dir to  /usr/hdp/{new-version}/knox/data

In the case of 2.3.0.0 -> 2.3.+, previous_data_dir is /var/lib/knox/data/
In the case of 2.3.+ -> 2.3.+, previous_data_dir is /usr/hdp/{prev-version}/knox/data


Diffs
-----

  ambari-common/src/main/python/resource_management/libraries/script/script.py e0a7877 
  ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
990fc87 
  ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/upgrade.py
2c805fa 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager_upgrade.py
9ac2d63 
  ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 3ae42a0 

Diff: https://reviews.apache.org/r/38543/diff/


Testing
-------

Ran python unit tests, which passed.
Verified during RU in several scenarios.
1. Fresh install of HDP 2.3.0.0 and RU to 2.3.2.0 with Knox. Before Finalizing, Downgraded.
2. Fresh install of HDP 2.3.0.0 and RU to 2.3.2.0 with Knox. Finalized.
3. Steps from #2, and then perform an RU to a higher build in HDP 2.3.2.0


Thanks,

Alejandro Fernandez


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message