helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jianqiu Lu <l...@yahoo-inc.com>
Subject Re: question on a state change scenario
Date Wed, 18 Jun 2014 18:30:17 GMT
Hi Kishore,Kanak,
  Thanks for your reply. In my case it is a participant-level property, an instance can have
different state for different partitions. Ex. Partition 1-3 are on instance 1, and partition
1 is read , partition 2 is weak read ,partition 3 is read.  It does not have a strong constrain
in my case, but updates ideal state may have some race conditions. From my understand, I need
to pass the whole ideal state(all partitions) as a map to the method, but what if two servers
update their ideal state at the same time?  By the way how does #2 work? Is there any example?
 For failure handling we have different replica consumes the same queue, and register both
of the two servers in different replica to same partition, so unless both of them are down,
we can always have some one reachable.


From: kishore g <g.kishore@gmail.com<mailto:g.kishore@gmail.com>>
Date: Tuesday, June 17, 2014 at 10:22 PM
To: "user@helix.apache.org<mailto:user@helix.apache.org>" <user@helix.apache.org<mailto:user@helix.apache.org>>,
Jianqiu Lv <lvjq@yahoo-inc.com<mailto:lvjq@yahoo-inc.com>>
Cc: Gavin Li <liyu@yahoo-inc.com<mailto:liyu@yahoo-inc.com>>
Subject: Re: question on a state change scenario

Hi Jianqiu,

Thank you for the kind words. I agree with what Kanak said but it mostly applies to systems
where state level constraint is important. For example, if the state model is master-slave,
we don't want the slave to automatically say that its the new master.

Looks like in your systems, there are no strong constraints on the state. In this case it
is ok for the participant to control its state. You have two options

#1. The participant can decide the state and update the idealstate to reflect that state.
This will be reflected in the ExternalView and the spectators can use the state to route requests.
You can do this if you don't intend to have any hard constraints on the state.

#2. Another option is to use the feature of requested state, the participant can request the
controller to go to a specific state. The controller however can decide if it can honor that

For the application you have described #1 can work. Let us know if #1 is acceptable.

Actually, I can see this as a requirement as a common pattern. We can allow the participants
to override the state in certain scenarios. We should be able to provide a much simpler api
to accomplish this.

 Also what is your requirement for failure handling.

Kishore G

On Tue, Jun 17, 2014 at 8:46 PM, Kanak Biscuitwala <kanak.b@hotmail.com<mailto:kanak.b@hotmail.com>>
Hi Jianqiu,

In Helix we typically discourage participants from changing the states of partitions they
serve because then we have multiple brains in the system.

It seems like the read property is a participant-level property rather than a partition-level
state. Is this true? Can an instance have different speeds per partition on the same instance?
If it is purely participant-level, then you can use HelixAdmin#setConfig to set this flag
for the participant, and you can register an InstanceConfigChangeListener with HelixManager
in order to listen for changes.

It is certainly possible to use HelixDataAccessor#updateIdealState to only update the partitions
that currently belong to an instance, but that feels more like a hack, and it requires you
to fully customize the ideal state calculation logic, rather than using FULL_AUTO or SEMI_AUTO


> From: lvjq@yahoo-inc.com<mailto:lvjq@yahoo-inc.com>
> To: user@helix.apache.org<mailto:user@helix.apache.org>
> CC: liyu@yahoo-inc.com<mailto:liyu@yahoo-inc.com>
> Subject: question on a state change scenario
> Date: Wed, 18 Jun 2014 02:05:00 +0000
> Hi team,
> First I want to thanks all for this great software.
> I am using helix for our storage system. In our system we have 3
> states read , weak read and offline. The state transactions are
> read-> <- weak read-> <-offline, very similar to the master/slave
> example. But the difference is that we need participate to control its
> status , which means a participate can decide what is itself's current
> ideal state. If a participate find he can’t follow write speed(we use a
> queue to do write , participate consumes message from this queue), then
> it can change its state from read to weak read. Is there a proper way
> to do this state change from participate side? I looked into Helix java
> doc and found in HelixAdmin there is a method “rebalance” seems can
> achieve my goal, but my scenario is only need to change one server’s
> state,“rebalance” seems too over. Or does this a Helix fit scenario?
> The other solution I thought about is using zookeeper to manage read
> and weak read states, and helix only maintain the online offline state.
> Once I get an online server from helix , then I will look up into
> zookeeper to find out what is the current read state of this
> server(maybe push mode from zookeeper is more proper). In this
> solution we don’t need to change helix ideal state from participate
> side, but this seems like mix helix and zookeeper to maintain state
> machine together. Could you please give me some advise on my thoughts?
> Thanks
> Jianqiu

View raw message