helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kishore g <g.kish...@gmail.com>
Subject Re: question on a state change scenario
Date Wed, 18 Jun 2014 20:02:43 GMT
Hi Jianqui,

For updating the idealstate, yes race conditions are possible. There is a
low level api that allows you to handle the race conditions.

see HelixDataAccessor.update method: This method does a merge of existing
and new values. For example, for a participant to go into read_only state
for a partition. It can do
is= new IdealState();
is.set("participant_id", "partition_id", "read_only");

This should handle the race conditions. Jason/Kanak, please correct me if I
am wrong.

Which version of Helix are you using. I think documentation for #2 is weak
and its something used internally by Helix for Task Management. Kanak, do
we have any example for this ?

Kishore G

On Wed, Jun 18, 2014 at 11:30 AM, Jianqiu Lu <lvjq@yahoo-inc.com> wrote:

>  Hi Kishore,Kanak,
>   Thanks for your reply. In my case it is a participant-level property, an
> instance can have different state for different partitions. Ex. Partition
> 1-3 are on instance 1, and partition 1 is read , partition 2 is weak read
> ,partition 3 is read.  It does not have a strong constrain in my case, but
> updates ideal state may have some race conditions. From my understand, I
> need to pass the whole ideal state(all partitions) as a map to the method,
> but what if two servers update their ideal state at the same time?  By the
> way how does #2 work? Is there any example?  For failure handling we have
> different replica consumes the same queue, and register both of the two
> servers in different replica to same partition, so unless both of them are
> down, we can always have some one reachable.
>  Thanks
> Jianqiu
>   From: kishore g <g.kishore@gmail.com>
> Date: Tuesday, June 17, 2014 at 10:22 PM
> To: "user@helix.apache.org" <user@helix.apache.org>, Jianqiu Lv <
> lvjq@yahoo-inc.com>
> Cc: Gavin Li <liyu@yahoo-inc.com>
> Subject: Re: question on a state change scenario
>     Hi Jianqiu,
>  Thank you for the kind words. I agree with what Kanak said but it mostly
> applies to systems where state level constraint is important. For example,
> if the state model is master-slave, we don't want the slave to
> automatically say that its the new master.
>  Looks like in your systems, there are no strong constraints on the state.
> In this case it is ok for the participant to control its state. You have
> two options
>  #1. The participant can decide the state and update the idealstate to
> reflect that state. This will be reflected in the ExternalView and the
> spectators can use the state to route requests. You can do this if you
> don't intend to have any hard constraints on the state.
>  #2. Another option is to use the feature of requested state, the
> participant can request the controller to go to a specific state. The
> controller however can decide if it can honor that request.
>  For the application you have described #1 can work. Let us know if #1 is
> acceptable.
>  Actually, I can see this as a requirement as a common pattern. We can
> allow the participants to override the state in certain scenarios. We
> should be able to provide a much simpler api to accomplish this.
>   Also what is your requirement for failure handling.
>  thanks,
> Kishore G
> On Tue, Jun 17, 2014 at 8:46 PM, Kanak Biscuitwala <kanak.b@hotmail.com>
> wrote:
>> Hi Jianqiu,
>> In Helix we typically discourage participants from changing the states of
>> partitions they serve because then we have multiple brains in the system.
>> It seems like the read property is a participant-level property rather
>> than a partition-level state. Is this true? Can an instance have different
>> speeds per partition on the same instance? If it is purely
>> participant-level, then you can use HelixAdmin#setConfig to set this flag
>> for the participant, and you can register an InstanceConfigChangeListener
>> with HelixManager in order to listen for changes.
>> It is certainly possible to use HelixDataAccessor#updateIdealState to
>> only update the partitions that currently belong to an instance, but that
>> feels more like a hack, and it requires you to fully customize the ideal
>> state calculation logic, rather than using FULL_AUTO or SEMI_AUTO mode.
>> Kanak
>> ________________________________
>> > From: lvjq@yahoo-inc.com
>> > To: user@helix.apache.org
>> > CC: liyu@yahoo-inc.com
>> > Subject: question on a state change scenario
>> > Date: Wed, 18 Jun 2014 02:05:00 +0000
>>  >
>> > Hi team,
>> > First I want to thanks all for this great software.
>> > I am using helix for our storage system. In our system we have 3
>> > states read , weak read and offline. The state transactions are
>> > read-> <- weak read-> <-offline, very similar to the master/slave
>> > example. But the difference is that we need participate to control its
>> > status , which means a participate can decide what is itself's current
>> > ideal state. If a participate find he can’t follow write speed(we use a
>> > queue to do write , participate consumes message from this queue), then
>> > it can change its state from read to weak read. Is there a proper way
>> > to do this state change from participate side? I looked into Helix java
>> > doc and found in HelixAdmin there is a method “rebalance” seems can
>> > achieve my goal, but my scenario is only need to change one server’s
>> > state,“rebalance” seems too over. Or does this a Helix fit scenario?
>> > The other solution I thought about is using zookeeper to manage read
>> > and weak read states, and helix only maintain the online offline state.
>> > Once I get an online server from helix , then I will look up into
>> > zookeeper to find out what is the current read state of this
>> > server(maybe push mode from zookeeper is more proper). In this
>> > solution we don’t need to change helix ideal state from participate
>> > side, but this seems like mix helix and zookeeper to maintain state
>> > machine together. Could you please give me some advise on my thoughts?
>> >
>> > Thanks
>> > Jianqiu

View raw message