zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Junqueira <fpjunque...@yahoo.com.INVALID>
Subject Re: zookeeper cluster change impact on kafka
Date Tue, 03 Nov 2015 16:47:02 GMT
I'm copying the Kafka user list here:

> We have a 3 node zookeeper cluster and a kafka cluster (3 nodes) using this
> zookeeper cluster. we want to migrate the zookeeper nodes to better boxes (
> hardware improvements). we already setup 3 new nodes.
> 
> can some one tell me what is the safe way to do the migration.
> 
> 1. what happens if we change the zookeeper connection string in kafka nodes
> to point to  3 new ZK nodes and start the kafka nodes one by one?

The new cluster will start with an empty state. Switching to a new cluster abruptly like you
suggest will mean loss of data, which is bad. Also, you won't be able to switch all brokers
at once, so for some time, you'll have brokers talking to different ensembles, which can lead
to an inconsistent state as well.

> 2. Do we need to change the old and new zookeeper nodes configuration before
> applying any changes on kafka nodes (zk clients)? like adding new nodes as
> observers etc...

You need to introduce one new node at a time. In the same step that you introduce a node,
you remove a node from the old ensemble. To do this, you need to change the server configuration
at each step to remove a server and add a server, and wait until a quorum follows a leader
before moving to the next step. This way you guarantee that the state of zookeeper in the
old ensemble migrates to the new ensemble.

> 3. what happens if we change the kafka configuration to talk to all 6 ZK
> Nodes and later shut down 3 old nodes and reconfigure the kafka to talk to 3
> new ZK nodes.

You can get into an inconsistent state if you do this. There is a chance that it works, but
I don't recommend it.

-Flavio

> 
> 
> 
> --
> View this message in context: http://zookeeper-user.578899.n2.nabble.com/zookeeper-cluster-change-impact-on-kafka-tp7581711.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.


Mime
View raw message