helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhen Zhang <zzh...@linkedin.com>
Subject RE: Changing the number of partitions in a resource
Date Tue, 17 Feb 2015 18:16:43 GMT
Hi Vinoth, you can use HelixAdmin#setResourceIdealState(), passing the new ideal-state.

Thanks,
Jason

________________________________
From: Vinoth Chandar [vinoth@uber.com]
Sent: Monday, February 16, 2015 5:54 PM
To: user
Subject: Re: Changing the number of partitions in a resource

Thanks for the response, Kishore.

I am basically mimicking the rabbitmq consumer group example..

helixAdmin.addResource(CLUSTER_NAME, RESOURCE_NAME, 10, STATE_MODEL_NAME,
      RebalanceMode.FULL_AUTO.toString)

So, I simple do another addResource(..) call with 15 instead of 10?

Thanks
Vinoth




On Mon, Feb 16, 2015 at 5:41 PM, kishore g <g.kishore@gmail.com<mailto:g.kishore@gmail.com>>
wrote:
Hi Vinoth,

The number of partitions can be changed dynamically. The changes required from your side depends
on idealstate mode.

FULL_AUTO
-- Simply change the number of partitions. Helix should assign the new partition to one of
the nodes automatically. Lets say you have 10 partitions, if you change the number of partitions
to 15. Partitions 11 to 15 will be distributed among the nodes.

-- SEMI_AUTO
-- In this case, you have to first change the number of partitions and invoke the rebalance
command. This will rebalance the partitions. There should be an option to use the existing
assignment so that shuffling of partitions is minimized.

-- CUSTOM
-- Change the ideal state either manually or dynamically using user defined rebalancer

Hope that helps
thanks,
Kishore G


On Mon, Feb 16, 2015 at 5:22 PM, Vinoth Chandar <vinoth@uber.com<mailto:vinoth@uber.com>>
wrote:
Hey guys,

I have a rather unusual case, where the number of partitions in my resource could change dynamically..
Is there a way to handle this without dropping and adding the resource again?

What kind of cleanup should I do if I drop and re-add? (e.g.: reinit state model objects?)..

Thanks
Vinoth





Mime
View raw message