ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez" <>
Subject Re: Review Request 40377: AMBARI-13918. Stack Upgrade: Kafka to call migration script when upgrading across HDP
Date Tue, 17 Nov 2015 20:51:43 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Nov. 17, 2015, 8:51 p.m.)

Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole,
and Swapan Shridhar.

Bugs: AMBARI-13918

Repository: ambari


In HDP, the the structure of stored acls for Kafka in zookeeper has changed. This
means to support upgrade-to or downgrade-from HDP we will have to move these stored
acls to new format or vice versa. 

Ambari should make the following changes for upgrade/downgrade process of Kafka.

PreReq: Install bits side-by-side

1) Stop all Kafka brokers
2) Call 
bin/ --authorizer --authorizer-properties
zookeeper.connect=localhost:2181 --upgradeAcls
bin/ --authorizer --authorizer-properties
zookeeper.connect=localhost:2181 --downgradeAcls 
3) Start all Kafka brokers.

Example Cases:
1) If upgrading from HDP ->, we will still stop all Kafka brokers since
cannot distinguish the version in the single upgrade pack. Customers may modify this behavior
by editing upgrade-2.3.xml.
Because the boundary is not crossed, we should not call the migration script.
2) If upgrading from HDP>, then we will stop all Kafka brokers and not
call the migration script since not crossing the boundary.
3) If upgrading from HDP>, then we will stop all Kafka brokers and call
the migration script since crossing the boundary. On downgrade, we should call the migration
script again.

Diffs (updated)

  ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml fe87ed2 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 90f50a4 



Tested in 

HDP 2.2 with Kafka, then EU from 2.2 -> called migration script, then added NameNode
HA, and RU from to did not call script
HDP 2.2 with kafka, then added NameNode HA, and RU from HDP 2.2->, which called
migration script, followed by RU from to which did not call script

Python unit tests passed,
Total run:829
Total errors:0
Total failures:0


Alejandro Fernandez

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message