helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hang Qi <hangq.1...@gmail.com>
Subject Re: Question regarding to CUSTOMIZED balance mode
Date Mon, 21 Sep 2015 19:56:14 GMT
Sure, I will first test the new rebalancer, and after that I can send out
the pull request, and  Iet's see whether it is generalized or not.

Thanks
Hang Qi

On Monday, September 21, 2015, kishore g <g.kishore@gmail.com> wrote:

> Please consider contributing the rebalancer to Helix if it can be
> generalized.
> On Sep 21, 2015 11:57 AM, "Hang Qi" <hangq.1985@gmail.com
> <javascript:_e(%7B%7D,'cvml','hangq.1985@gmail.com');>> wrote:
>
>> Hi Kanak & Kishore,
>>
>> Thanks very much for your reply.
>>
>> Yes, I have looked into the HelixCustomCodeRunner by myself, however I
>> ended up to go with option 2, write our own rebalancer.
>>
>> Our use case is that we want to extend the Master/Slave state model by
>> adding one special state for special purpose, like partition splitting.
>> Normally, partitions still follow Master/Slave model to perform the state
>> transition. And we have a admin node, which can mark some partitions on one
>> instance to this special state in IdealState before the operation, and
>> after the operation is done, the admin node can remove the special state
>> for those partitions on the instance in IdealState.
>>
>> We are using SEMI_AUTO mode right now, so we are looking into the CUSTOM
>> mode, later on, we think the our new rebalancer is kind of mixing of
>> SEMI_AUTO and CUSTOM, thus we go with option 2.
>>
>> Thanks
>> Hang Qi
>>
>> On Sat, Sep 19, 2015 at 2:04 PM, kishore g <g.kishore@gmail.com
>> <javascript:_e(%7B%7D,'cvml','g.kishore@gmail.com');>> wrote:
>>
>>> There are two ways to  write your own rebalancer
>>>
>>> #1. Using CustomCode Runner. This is regsistered in any of the
>>> participants and can be invoked on any node in the cluster starts/stops.
>>> Kanak's email points to this option.
>>>
>>> #2. Implement the rebalancer interface. This is one you have specified
>>> in your email. This is run in controller and is invoked on each and every
>>> change in the cluster such as node added/removed, start/stop, resource
>>> add/remove etc.
>>>
>>> Let me know which mode you are planning to use, we can provide
>>> additional details.
>>>
>>>
>>>
>>> On Sat, Sep 19, 2015 at 11:19 AM, Kanak Biscuitwala <kanak.b@hotmail.com
>>> <javascript:_e(%7B%7D,'cvml','kanak.b@hotmail.com');>> wrote:
>>>
>>>> Hi,
>>>>
>>>>
>>>> Yeah, you need to register a custom code runner. Here is an example:
>>>> https://github.com/apache/helix/blob/master/helix-core/src/test/java/org/apache/helix/integration/TestHelixCustomCodeRunner.java
>>>>
>>>>
>>>>
>>>> Kanak
>>>>
>>>> From: hangq.1985@gmail.com
>>>> <javascript:_e(%7B%7D,'cvml','hangq.1985@gmail.com');>
>>>> Date: Mon, 14 Sep 2015 23:04:21 -0700
>>>> Subject: Question regarding to CUSTOMIZED balance mode
>>>> To: user@helix.apache.org
>>>> <javascript:_e(%7B%7D,'cvml','user@helix.apache.org');>
>>>>
>>>> Hi,
>>>>
>>>> We are evaluating CUSTOMIZED balance mode in helix 0.6.4 currently, and
>>>> I have one question regarding to it.
>>>>
>>>> How do I handle cluster failure case, e.g. the initial ideal state as
>>>> following:
>>>> "mapFields" : {
>>>>    "MyResource_0" : {
>>>>      "N1" : "MASTER",
>>>>      "N2" : "SLAVE",
>>>>    }
>>>> }
>>>> If N1 goes down, N2 will not become MASTER automatically, right?
>>>>
>>>> If so, how do I register the listener for cluster state change to
>>>> update ideal state? The doc says "The application needs to implement a
>>>> callback interface that Helix invokes when the cluster state changes.
>>>> Within this callback, the application can recompute the idealstate.", is
>>>> that interface Rebalancer?
>>>>
>>>> Do we have an example for CUSTOMIZED balance?
>>>>
>>>> Besides, when looking at the code, CustomRebalancer implements
>>>> Rebalancer interface, and just returns currentIdealState
>>>> when computeNewIdealState is called.
>>>>
>>>> Thanks
>>>> Hang Qi
>>>>
>>>
>>>
>>
>>
>>
>>

-- 
Qi hang

Mime
View raw message