ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez" <afernan...@hortonworks.com>
Subject Re: Review Request 40377: AMBARI-13918. Stack Upgrade: Kafka to call migration script when upgrading across HDP 2.3.4.0
Date Tue, 17 Nov 2015 20:47:55 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40377/#review106934
-----------------------------------------------------------



ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml (lines 787 - 794)
<https://reviews.apache.org/r/40377/#comment165738>

    Still called during downgrade.
    pre-upgrade just happens before restart, in any direction.


- Alejandro Fernandez


On Nov. 17, 2015, 2:50 a.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40377/
> -----------------------------------------------------------
> 
> (Updated Nov. 17, 2015, 2:50 a.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Nate
Cole, and Swapan Shridhar.
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> In HDP 2.3.4.0, the the structure of stored acls for Kafka in zookeeper has changed.
This means to support upgrade-to or downgrade-from HDP 2.3.4.0 we will have to move these
stored acls to new format or vice versa. 
> 
> Ambari should make the following changes for upgrade/downgrade process of Kafka.
> 
> PreReq: Install bits side-by-side
> 
> Steps:
> 1) Stop all Kafka brokers
> 2) Call 
> bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
zookeeper.connect=localhost:2181 --upgradeAcls
> or
> bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
zookeeper.connect=localhost:2181 --downgradeAcls 
> 3) Start all Kafka brokers.
> 
> Example Cases:
> 1) If upgrading from HDP 2.3.0.0 -> 2.3.2.0, we will still stop all Kafka brokers
since cannot distinguish the version in the single upgrade pack. Customers may modify this
behavior by editing upgrade-2.3.xml.
> Because the 2.3.4.0 boundary is not crossed, we should not call the migration script.
> 2) If upgrading from HDP 2.3.4.0->2.3.5.0, then we will stop all Kafka brokers and
not call the migration script since not crossing the boundary.
> 3) If upgrading from HDP 2.3.2.0->2.3.5.0, then we will stop all Kafka brokers and
call the migration script since crossing the boundary. On downgrade, we should call the migration
script again.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka_broker.py
8efd564 
>   ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py
dc3a61b 
>   ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/upgrade.py
22036f7 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml fe87ed2 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 90f50a4 
> 
> Diff: https://reviews.apache.org/r/40377/diff/
> 
> 
> Testing
> -------
> 
> Tested in 
> 
> HDP 2.2 with Kafka, then EU from 2.2 -> 2.3.4.0 called migration script, then added
NameNode HA, and RU from 2.3.4.0-1 to 2.3.4.0-2 did not call script
> HDP 2.2 with kafka, then added NameNode HA, and RU from HDP 2.2->2.3.4.0-1, which
called migration script, followed by RU from 2.3.4.0-1 to 2.3.4.0-2 which did not call script
> 
> Python unit tests passed,
> ----------------------------------------------------------------------
> Total run:829
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message