helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kanak Biscuitwala <kana...@hotmail.com>
Subject RE: question on a state change scenario
Date Wed, 18 Jun 2014 20:04:08 GMT

Hi Jianqiu,

You need to use HelixDataAccessor#update to ensure atomicity, like:

HelixManager manager = ...;
HelixDataAccessor accessor = manager.getHelixDataAccessor();
IdealState delta = new IdealState(resourceName);
delta.getRecord().getMapFields().put(partitionName, newState);
accessor.updateProperty(accessor.keyBuilder().idealStates(resourceName), delta);

Kanak
________________________________
> From: lvjq@yahoo-inc.com 
> To: g.kishore@gmail.com; user@helix.apache.org 
> CC: liyu@yahoo-inc.com 
> Subject: Re: question on a state change scenario 
> Date: Wed, 18 Jun 2014 18:30:17 +0000 
> 
> Hi Kishore,Kanak, 
> Thanks for your reply. In my case it is a participant-level property, 
> an instance can have different state for different partitions. Ex. 
> Partition 1-3 are on instance 1, and partition 1 is read , partition 2 
> is weak read ,partition 3 is read. It does not have a strong constrain 
> in my case, but updates ideal state may have some race conditions. From 
> my understand, I need to pass the whole ideal state(all partitions) as 
> a map to the method, but what if two servers update their ideal state 
> at the same time? By the way how does #2 work? Is there any example? 
> For failure handling we have different replica consumes the same 
> queue, and register both of the two servers in different replica to 
> same partition, so unless both of them are down, we can always have 
> some one reachable. 
> 
> Thanks 
> Jianqiu 
> 
> From: kishore g <g.kishore@gmail.com<mailto:g.kishore@gmail.com>> 
> Date: Tuesday, June 17, 2014 at 10:22 PM 
> To: "user@helix.apache.org<mailto:user@helix.apache.org>" 
> <user@helix.apache.org<mailto:user@helix.apache.org>>, Jianqiu Lv 
> <lvjq@yahoo-inc.com<mailto:lvjq@yahoo-inc.com>> 
> Cc: Gavin Li <liyu@yahoo-inc.com<mailto:liyu@yahoo-inc.com>> 
> Subject: Re: question on a state change scenario 
> 
> Hi Jianqiu, 
> 
> Thank you for the kind words. I agree with what Kanak said but it 
> mostly applies to systems where state level constraint is important. 
> For example, if the state model is master-slave, we don't want the 
> slave to automatically say that its the new master. 
> 
> Looks like in your systems, there are no strong constraints on the 
> state. In this case it is ok for the participant to control its state. 
> You have two options 
> 
> #1. The participant can decide the state and update the idealstate to 
> reflect that state. This will be reflected in the ExternalView and the 
> spectators can use the state to route requests. You can do this if you 
> don't intend to have any hard constraints on the state. 
> 
> #2. Another option is to use the feature of requested state, the 
> participant can request the controller to go to a specific state. The 
> controller however can decide if it can honor that request. 
> 
> For the application you have described #1 can work. Let us know if #1 
> is acceptable. 
> 
> Actually, I can see this as a requirement as a common pattern. We can 
> allow the participants to override the state in certain scenarios. We 
> should be able to provide a much simpler api to accomplish this. 
> 
> Also what is your requirement for failure handling. 
> 
> thanks, 
> Kishore G 
> 
> 
> 
> On Tue, Jun 17, 2014 at 8:46 PM, Kanak Biscuitwala 
> <kanak.b@hotmail.com<mailto:kanak.b@hotmail.com>> wrote: 
> Hi Jianqiu, 
> 
> In Helix we typically discourage participants from changing the states 
> of partitions they serve because then we have multiple brains in the 
> system. 
> 
> It seems like the read property is a participant-level property rather 
> than a partition-level state. Is this true? Can an instance have 
> different speeds per partition on the same instance? If it is purely 
> participant-level, then you can use HelixAdmin#setConfig to set this 
> flag for the participant, and you can register an 
> InstanceConfigChangeListener with HelixManager in order to listen for 
> changes. 
> 
> It is certainly possible to use HelixDataAccessor#updateIdealState to 
> only update the partitions that currently belong to an instance, but 
> that feels more like a hack, and it requires you to fully customize the 
> ideal state calculation logic, rather than using FULL_AUTO or SEMI_AUTO 
> mode. 
> 
> Kanak 
> 
> ________________________________ 
>> From: lvjq@yahoo-inc.com<mailto:lvjq@yahoo-inc.com> 
>> To: user@helix.apache.org<mailto:user@helix.apache.org> 
>> CC: liyu@yahoo-inc.com<mailto:liyu@yahoo-inc.com> 
>> Subject: question on a state change scenario 
>> Date: Wed, 18 Jun 2014 02:05:00 +0000 
>> 
>> Hi team, 
>> First I want to thanks all for this great software. 
>> I am using helix for our storage system. In our system we have 3 
>> states read , weak read and offline. The state transactions are 
>> read-> <- weak read-> <-offline, very similar to the master/slave 
>> example. But the difference is that we need participate to control its 
>> status , which means a participate can decide what is itself's current 
>> ideal state. If a participate find he can’t follow write speed(we use a 
>> queue to do write , participate consumes message from this queue), then 
>> it can change its state from read to weak read. Is there a proper way 
>> to do this state change from participate side? I looked into Helix java 
>> doc and found in HelixAdmin there is a method “rebalance” seems can 
>> achieve my goal, but my scenario is only need to change one server’s 
>> state,“rebalance” seems too over. Or does this a Helix fit scenario? 
>> The other solution I thought about is using zookeeper to manage read 
>> and weak read states, and helix only maintain the online offline state. 
>> Once I get an online server from helix , then I will look up into 
>> zookeeper to find out what is the current read state of this 
>> server(maybe push mode from zookeeper is more proper). In this 
>> solution we don’t need to change helix ideal state from participate 
>> side, but this seems like mix helix and zookeeper to maintain state 
>> machine together. Could you please give me some advise on my thoughts? 
>> 
>> Thanks 
>> Jianqiu 
> 
> 
 		 	   		  
Mime
View raw message