helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kishore g <g.kish...@gmail.com>
Subject Re: Question regarding to CUSTOMIZED balance mode
Date Mon, 21 Sep 2015 19:21:21 GMT
Please consider contributing the rebalancer to Helix if it can be
generalized.
On Sep 21, 2015 11:57 AM, "Hang Qi" <hangq.1985@gmail.com> wrote:

> Hi Kanak & Kishore,
>
> Thanks very much for your reply.
>
> Yes, I have looked into the HelixCustomCodeRunner by myself, however I
> ended up to go with option 2, write our own rebalancer.
>
> Our use case is that we want to extend the Master/Slave state model by
> adding one special state for special purpose, like partition splitting.
> Normally, partitions still follow Master/Slave model to perform the state
> transition. And we have a admin node, which can mark some partitions on one
> instance to this special state in IdealState before the operation, and
> after the operation is done, the admin node can remove the special state
> for those partitions on the instance in IdealState.
>
> We are using SEMI_AUTO mode right now, so we are looking into the CUSTOM
> mode, later on, we think the our new rebalancer is kind of mixing of
> SEMI_AUTO and CUSTOM, thus we go with option 2.
>
> Thanks
> Hang Qi
>
> On Sat, Sep 19, 2015 at 2:04 PM, kishore g <g.kishore@gmail.com> wrote:
>
>> There are two ways to  write your own rebalancer
>>
>> #1. Using CustomCode Runner. This is regsistered in any of the
>> participants and can be invoked on any node in the cluster starts/stops.
>> Kanak's email points to this option.
>>
>> #2. Implement the rebalancer interface. This is one you have specified in
>> your email. This is run in controller and is invoked on each and every
>> change in the cluster such as node added/removed, start/stop, resource
>> add/remove etc.
>>
>> Let me know which mode you are planning to use, we can provide additional
>> details.
>>
>>
>>
>> On Sat, Sep 19, 2015 at 11:19 AM, Kanak Biscuitwala <kanak.b@hotmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>>
>>> Yeah, you need to register a custom code runner. Here is an example:
>>> https://github.com/apache/helix/blob/master/helix-core/src/test/java/org/apache/helix/integration/TestHelixCustomCodeRunner.java
>>>
>>>
>>>
>>> Kanak
>>>
>>> From: hangq.1985@gmail.com
>>> Date: Mon, 14 Sep 2015 23:04:21 -0700
>>> Subject: Question regarding to CUSTOMIZED balance mode
>>> To: user@helix.apache.org
>>>
>>> Hi,
>>>
>>> We are evaluating CUSTOMIZED balance mode in helix 0.6.4 currently, and
>>> I have one question regarding to it.
>>>
>>> How do I handle cluster failure case, e.g. the initial ideal state as
>>> following:
>>> "mapFields" : {
>>>    "MyResource_0" : {
>>>      "N1" : "MASTER",
>>>      "N2" : "SLAVE",
>>>    }
>>> }
>>> If N1 goes down, N2 will not become MASTER automatically, right?
>>>
>>> If so, how do I register the listener for cluster state change to update
>>> ideal state? The doc says "The application needs to implement a callback
>>> interface that Helix invokes when the cluster state changes. Within this
>>> callback, the application can recompute the idealstate.", is that interface
>>> Rebalancer?
>>>
>>> Do we have an example for CUSTOMIZED balance?
>>>
>>> Besides, when looking at the code, CustomRebalancer implements
>>> Rebalancer interface, and just returns currentIdealState
>>> when computeNewIdealState is called.
>>>
>>> Thanks
>>> Hang Qi
>>>
>>
>>
>
>
>
>

Mime
View raw message