helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kanak Biscuitwala <kana...@hotmail.com>
Subject RE: question on a state change scenario
Date Wed, 18 Jun 2014 20:06:16 GMT
The requested state feature is currently only supported in the task framework and otherwise
ignored. Currently the only way for a participant to change its own state is to update the
ideal state itself.


> Date: Wed, 18 Jun 2014 13:02:43 -0700 
> Subject: Re: question on a state change scenario 
> From: g.kishore@gmail.com 
> To: lvjq@yahoo-inc.com 
> CC: user@helix.apache.org; liyu@yahoo-inc.com 
> Hi Jianqui, 
> For updating the idealstate, yes race conditions are possible. There is 
> a low level api that allows you to handle the race conditions. 
> see HelixDataAccessor.update method: This method does a merge of 
> existing and new values. For example, for a participant to go into 
> read_only state for a partition. It can do 
> acce 
> is= new IdealState(); 
> is.set("participant_id", "partition_id", "read_only"); 
> helixdataaccessor.update(accessor.keyBuilder.idealState("resource_name"),is); 
> This should handle the race conditions. Jason/Kanak, please correct me 
> if I am wrong. 
> Which version of Helix are you using. I think documentation for #2 is 
> weak and its something used internally by Helix for Task Management. 
> Kanak, do we have any example for this ? 
> thanks, 
> Kishore G 
> On Wed, Jun 18, 2014 at 11:30 AM, Jianqiu Lu 
> <lvjq@yahoo-inc.com<mailto:lvjq@yahoo-inc.com>> wrote: 
> Hi Kishore,Kanak, 
> Thanks for your reply. In my case it is a participant-level property, 
> an instance can have different state for different partitions. Ex. 
> Partition 1-3 are on instance 1, and partition 1 is read , partition 2 
> is weak read ,partition 3 is read. It does not have a strong constrain 
> in my case, but updates ideal state may have some race conditions. From 
> my understand, I need to pass the whole ideal state(all partitions) as 
> a map to the method, but what if two servers update their ideal state 
> at the same time? By the way how does #2 work? Is there any example? 
> For failure handling we have different replica consumes the same 
> queue, and register both of the two servers in different replica to 
> same partition, so unless both of them are down, we can always have 
> some one reachable. 
> Thanks 
> Jianqiu 
> From: kishore g <g.kishore@gmail.com<mailto:g.kishore@gmail.com>> 
> Date: Tuesday, June 17, 2014 at 10:22 PM 
> To: "user@helix.apache.org<mailto:user@helix.apache.org>" 
> <user@helix.apache.org<mailto:user@helix.apache.org>>, Jianqiu Lv 
> <lvjq@yahoo-inc.com<mailto:lvjq@yahoo-inc.com>> 
> Cc: Gavin Li <liyu@yahoo-inc.com<mailto:liyu@yahoo-inc.com>> 
> Subject: Re: question on a state change scenario 
> Hi Jianqiu, 
> Thank you for the kind words. I agree with what Kanak said but it 
> mostly applies to systems where state level constraint is important. 
> For example, if the state model is master-slave, we don't want the 
> slave to automatically say that its the new master. 
> Looks like in your systems, there are no strong constraints on the 
> state. In this case it is ok for the participant to control its state. 
> You have two options 
> #1. The participant can decide the state and update the idealstate to 
> reflect that state. This will be reflected in the ExternalView and the 
> spectators can use the state to route requests. You can do this if you 
> don't intend to have any hard constraints on the state. 
> #2. Another option is to use the feature of requested state, the 
> participant can request the controller to go to a specific state. The 
> controller however can decide if it can honor that request. 
> For the application you have described #1 can work. Let us know if #1 
> is acceptable. 
> Actually, I can see this as a requirement as a common pattern. We can 
> allow the participants to override the state in certain scenarios. We 
> should be able to provide a much simpler api to accomplish this. 
> Also what is your requirement for failure handling. 
> thanks, 
> Kishore G 
> On Tue, Jun 17, 2014 at 8:46 PM, Kanak Biscuitwala 
> <kanak.b@hotmail.com<mailto:kanak.b@hotmail.com>> wrote: 
> Hi Jianqiu, 
> In Helix we typically discourage participants from changing the states 
> of partitions they serve because then we have multiple brains in the 
> system. 
> It seems like the read property is a participant-level property rather 
> than a partition-level state. Is this true? Can an instance have 
> different speeds per partition on the same instance? If it is purely 
> participant-level, then you can use HelixAdmin#setConfig to set this 
> flag for the participant, and you can register an 
> InstanceConfigChangeListener with HelixManager in order to listen for 
> changes. 
> It is certainly possible to use HelixDataAccessor#updateIdealState to 
> only update the partitions that currently belong to an instance, but 
> that feels more like a hack, and it requires you to fully customize the 
> ideal state calculation logic, rather than using FULL_AUTO or SEMI_AUTO 
> mode. 
> Kanak 
> ________________________________ 
>> From: lvjq@yahoo-inc.com<mailto:lvjq@yahoo-inc.com> 
>> To: user@helix.apache.org<mailto:user@helix.apache.org> 
>> CC: liyu@yahoo-inc.com<mailto:liyu@yahoo-inc.com> 
>> Subject: question on a state change scenario 
>> Date: Wed, 18 Jun 2014 02:05:00 +0000 
>> Hi team, 
>> First I want to thanks all for this great software. 
>> I am using helix for our storage system. In our system we have 3 
>> states read , weak read and offline. The state transactions are 
>> read-> <- weak read-> <-offline, very similar to the master/slave 
>> example. But the difference is that we need participate to control its 
>> status , which means a participate can decide what is itself's current 
>> ideal state. If a participate find he can’t follow write speed(we use a 
>> queue to do write , participate consumes message from this queue), then 
>> it can change its state from read to weak read. Is there a proper way 
>> to do this state change from participate side? I looked into Helix java 
>> doc and found in HelixAdmin there is a method “rebalance” seems can 
>> achieve my goal, but my scenario is only need to change one server’s 
>> state,“rebalance” seems too over. Or does this a Helix fit scenario? 
>> The other solution I thought about is using zookeeper to manage read 
>> and weak read states, and helix only maintain the online offline state. 
>> Once I get an online server from helix , then I will look up into 
>> zookeeper to find out what is the current read state of this 
>> server(maybe push mode from zookeeper is more proper). In this 
>> solution we don’t need to change helix ideal state from participate 
>> side, but this seems like mix helix and zookeeper to maintain state 
>> machine together. Could you please give me some advise on my thoughts? 
>> Thanks 
>> Jianqiu 
View raw message