helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kanak Biscuitwala <kana...@hotmail.com>
Subject RE: New rebalancers: deltarebalancer & rebalancer
Date Thu, 26 Sep 2013 23:54:08 GMT
Well actually I guess I can answer my own question: the current rebalancer interface seems
to be general enough to support adding a new delta implementation that would fit in the current
pipeline.

----------------------------------------
> From: kanak.b@hotmail.com
> To: user@helix.incubator.apache.org; dev@helix.incubator.apache.org
> Subject: RE: New rebalancers: deltarebalancer & rebalancer
> Date: Thu, 26 Sep 2013 16:51:42 -0700
>
> Seems reasonable. I guess this doesn't fit into a traditional state model because the
rebalancer can generate a new "state" each time. Do you envision this being a generalization
of the current rebalancer, or as part of a new pipeline?
>
> Kanak
> ________________________________
>> Date: Thu, 26 Sep 2013 10:53:44 -0700
>> Subject: New rebalancers: deltarebalancer & rebalancer
>> From: g.kishore@gmail.com
>> To: dev@helix.incubator.apache.org; user@helix.incubator.apache.org
>>
>> Hi,
>>
>> I am thinking of adding a delta rebalancer that simple compares the
>> delta between idealstate and currentstate and generate transitions for
>> the delta. The main difference from existing implementation is that
>> this does not need a state machine or any constraints on the state.
>>
>> Where can this be useful
>> One scenario where this can be useful is managing the config versions.
>> For example we can say
>>
>> Node1:1.3
>> Node2:1.3
>>
>> Each version can correspond to a set a config properties. If a new
>> config is uploaded with new version, we send the transition for each
>> node to refresh the config. The benefit of this is clients will know
>> what is the config version the participant is running. And leader can
>> ensure that all participants are upgraded to latest config.
>>
>> We can take this a step further and allow the idealstate to be more
>> expressive
>>
>> for example the state can be composed of multiple attributes
>>
>> N1: {a: aval, b:bval, c:cval}
>> N1: {a: aval, b:bval, c:cval}
>>
>>
>> And we can change each value independently.
>>
>> N1:{a:avalnew, b:bval, c:cval}
>>
>> we could then invoke the appropriate call back on the participant that
>> a changed and once the callback is processed the current state changes.
>>
>> This kind of becomes a general way to manage configuration and keep
>> track of what config the node is using. Note this is not supposed to
>> say every single config property but it could be used to represent high
>> level config properties.
>>
>>
>> Thoughts/feedback? At LinkedIn, i think this will help us manage schema
>> versions.
>>
>> Thanks,
>> Kishore G
>>
>>
>>
>>
>>
>>
>>
>>
>> 		 	   		  
Mime
View raw message