helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hang Qi <hangq.1...@gmail.com>
Subject Re: Question regarding to CUSTOMIZED balance mode
Date Mon, 21 Sep 2015 18:57:23 GMT
Hi Kanak & Kishore,

Thanks very much for your reply.

Yes, I have looked into the HelixCustomCodeRunner by myself, however I
ended up to go with option 2, write our own rebalancer.

Our use case is that we want to extend the Master/Slave state model by
adding one special state for special purpose, like partition splitting.
Normally, partitions still follow Master/Slave model to perform the state
transition. And we have a admin node, which can mark some partitions on one
instance to this special state in IdealState before the operation, and
after the operation is done, the admin node can remove the special state
for those partitions on the instance in IdealState.

We are using SEMI_AUTO mode right now, so we are looking into the CUSTOM
mode, later on, we think the our new rebalancer is kind of mixing of
SEMI_AUTO and CUSTOM, thus we go with option 2.

Thanks
Hang Qi

On Sat, Sep 19, 2015 at 2:04 PM, kishore g <g.kishore@gmail.com> wrote:

> There are two ways to  write your own rebalancer
>
> #1. Using CustomCode Runner. This is regsistered in any of the
> participants and can be invoked on any node in the cluster starts/stops.
> Kanak's email points to this option.
>
> #2. Implement the rebalancer interface. This is one you have specified in
> your email. This is run in controller and is invoked on each and every
> change in the cluster such as node added/removed, start/stop, resource
> add/remove etc.
>
> Let me know which mode you are planning to use, we can provide additional
> details.
>
>
>
> On Sat, Sep 19, 2015 at 11:19 AM, Kanak Biscuitwala <kanak.b@hotmail.com>
> wrote:
>
>> Hi,
>>
>>
>> Yeah, you need to register a custom code runner. Here is an example:
>> https://github.com/apache/helix/blob/master/helix-core/src/test/java/org/apache/helix/integration/TestHelixCustomCodeRunner.java
>>
>>
>>
>> Kanak
>>
>> From: hangq.1985@gmail.com
>> Date: Mon, 14 Sep 2015 23:04:21 -0700
>> Subject: Question regarding to CUSTOMIZED balance mode
>> To: user@helix.apache.org
>>
>> Hi,
>>
>> We are evaluating CUSTOMIZED balance mode in helix 0.6.4 currently, and I
>> have one question regarding to it.
>>
>> How do I handle cluster failure case, e.g. the initial ideal state as
>> following:
>> "mapFields" : {
>>    "MyResource_0" : {
>>      "N1" : "MASTER",
>>      "N2" : "SLAVE",
>>    }
>> }
>> If N1 goes down, N2 will not become MASTER automatically, right?
>>
>> If so, how do I register the listener for cluster state change to update
>> ideal state? The doc says "The application needs to implement a callback
>> interface that Helix invokes when the cluster state changes. Within this
>> callback, the application can recompute the idealstate.", is that interface
>> Rebalancer?
>>
>> Do we have an example for CUSTOMIZED balance?
>>
>> Besides, when looking at the code, CustomRebalancer implements Rebalancer
>> interface, and just returns currentIdealState when computeNewIdealState is
>> called.
>>
>> Thanks
>> Hang Qi
>>
>
>

Mime
View raw message