helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Co Ting Keh <la...@box.com>
Subject Re: calling admin.rebalance
Date Wed, 03 Jul 2013 17:40:34 GMT
Okay thank you. This means there is no harm to calling it every time a new
controller or participant is added correct? It would just reset the number
of replicas but will not cause a remapping. This is so that i do not have
to differentiate whether or not the  current run is the first or the nth.


Sincerely,
Lance


On Wed, Jul 3, 2013 at 10:07 AM, kishore g <g.kishore@gmail.com> wrote:

> Thanks Lance, that is correct. In case of AUTO_REBALANCE admin.rebalance
> only sets the number of replica.
>
>
> thanks,
> Kishore G
>
>
> On Wed, Jul 3, 2013 at 9:36 AM, Lance Co Ting Keh <lance@box.com> wrote:
>
>> Thanks for the response.
>>
>> Yes, but I think there is a small caveat. Even in AUTO_REBALANCE mode,
>> rebalance must be called once in the very beginning before any mapping to
>> occur. This indicates how many replicas is necessary per resource. As in
>> the lock manager demo not calling rebalance on line 68 (
>> http://helix.incubator.apache.org/recipes/lock_manager.html) causes no
>> initial mapping to occur.
>>
>>
>> Sincerely,
>> Lance
>>
>>
>> On Tue, Jul 2, 2013 at 10:19 PM, kishore g <g.kishore@gmail.com> wrote:
>>
>>> Hi Lance,
>>>
>>> The impact of calling rebalance depends on the ideal state execution
>>> mode.( AUTO_REBALANCE, AUTO, CUSTOM).
>>> http://helix.incubator.apache.org/tutorial_rebalance.html
>>>
>>> Rebalance is simply a mechanism of mapping partition to a node and
>>> state.
>>>
>>> AUTO_REBALANCE
>>> ------------------------------
>>> In this mode the controller automatically does the rebalancing when new
>>> participants are added and started/failed which means there is not need to
>>> call admin.rebalance every time a new participant is added.
>>>
>>> AUTO
>>> =====
>>> In this mode, controller will not move the partitions automatically. It
>>> only changes the state. Calling admin.rebalance every time a participant is
>>> added will cause lot of movement. For example, lets say you have 20 node
>>> cluster and you plan to add 10 nodes. To minimize the delta caused by
>>> re-mapping its better to call admin.rebalance after adding all nodes
>>> instead of calling it every time a node is added. Also note that
>>> admin.rebalance takes the current idealstate as an additional parameter.
>>> Helix comes with built in algorithm to minimize the movement.
>>>
>>>
>>> CUSTOM
>>> =======
>>> admin.rebalance is a no-op. The application needs to set the idealstate
>>> explicitly.
>>>
>>> Hope this helps.
>>>
>>>
>>>
>>> On Tue, Jul 2, 2013 at 9:33 PM, Lance Co Ting Keh <lance@box.com> wrote:
>>>
>>>> I have a quick implementation question. How often is admin.reblance
>>>> meant to be called? I know at the dawn of time, when the first controller
>>>> and participant is added onto an empty cluster, admin.rebalance needs to
be
>>>> called before the controller assigns the initial mapping. After this, new
>>>> controllers/participants are added to the cluster know how to handle
>>>> themselves, controllers become slave to an existing master controller (if
>>>> there is one) and the cluster is rebalanced to include the new participant.
>>>> I'm assuming the master controller automatically does this rebalancing, so
>>>> rebalance need not be called when adding succeeding participants.So should
>>>> admin.rebalance only be called at the dawn of time? Is there harm to
>>>> calling it every time a new participant is added--will this cause too many
>>>> remaps?
>>>>
>>>>
>>>> Thank you
>>>> Lance
>>>>
>>>
>>>
>>
>

Mime
View raw message