helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kanak Biscuitwala <kana...@hotmail.com>
Subject RE: New rebalancers: deltarebalancer & rebalancer
Date Thu, 26 Sep 2013 23:51:42 GMT
Seems reasonable. I guess this doesn't fit into a traditional state model because the rebalancer
can generate a new "state" each time. Do you envision this being a generalization of the current
rebalancer, or as part of a new pipeline?

Kanak
________________________________
> Date: Thu, 26 Sep 2013 10:53:44 -0700 
> Subject: New rebalancers: deltarebalancer & rebalancer 
> From: g.kishore@gmail.com 
> To: dev@helix.incubator.apache.org; user@helix.incubator.apache.org 
> 
> Hi, 
> 
> I am thinking of adding a delta rebalancer that simple compares the 
> delta between idealstate and currentstate and generate transitions for 
> the delta. The main difference from existing implementation is that 
> this does not need a state machine or any constraints on the state. 
> 
> Where can this be useful 
> One scenario where this can be useful is managing the config versions. 
> For example we can say 
> 
> Node1:1.3 
> Node2:1.3 
> 
> Each version can correspond to a set a config properties. If a new 
> config is uploaded with new version, we send the transition for each 
> node to refresh the config. The benefit of this is clients will know 
> what is the config version the participant is running. And leader can 
> ensure that all participants are upgraded to latest config. 
> 
> We can take this a step further and allow the idealstate to be more 
> expressive 
> 
> for example the state can be composed of multiple attributes 
> 
> N1: {a: aval, b:bval, c:cval} 
> N1: {a: aval, b:bval, c:cval} 
> 
> 
> And we can change each value independently. 
> 
> N1:{a:avalnew, b:bval, c:cval} 
> 
> we could then invoke the appropriate call back on the participant that 
> a changed and once the callback is processed the current state changes. 
> 
> This kind of becomes a general way to manage configuration and keep 
> track of what config the node is using. Note this is not supposed to 
> say every single config property but it could be used to represent high 
> level config properties. 
> 
> 
> Thoughts/feedback? At LinkedIn, i think this will help us manage schema 
> versions. 
> 
> Thanks, 
> Kishore G 
> 
> 
> 
> 
> 
> 
> 
> 
> 		 	   		  
Mime
View raw message