helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kishore g <g.kish...@gmail.com>
Subject Re: A few questions about helix.
Date Sun, 24 Feb 2013 16:45:06 GMT
3) Regarding overhead in case of too many spectators.
Do you mean over head in terms of  controller informing the spectator ?
 Controller does not communicate directly with the spectator. All
communication is via zookeeper. Its more like a push/pull model where
controller pushes to ZK and spectators pull from ZK. This is an important
difference from other systems where controllers communicate directly with
other  components in the system. This allows us to scale the system and not
be bottle necked by controller. Eventually ZK might be a bottle neck but
for spectators we can easily scale reads on ZK by adding more ZK observers.
In fact, if the system as lot of spectators its better to connect only to
ZK observers. Apart from Helix has group commit feature where transitions
are grouped together where reduces the number of notifications to

2) We dont have the feature to wait for configurable time before selecting
another slave partition. We have been asked for this feature many times, we
should probably add it :-).  However, we do have another feature which
might actually be useful and more elegant. You can pause/unpause the
controller. When the controller is paused no transitions will occur in the
system. Is this something that will be useful? The pause/unpause is a
cluster level.

1) CustoomcodeInvoker example

regarding LEADER_ELECTION state model. I see what you mean, this is
actually a very nice and cool idea. I got the part until all participants
getting into LEADER_ELECTION state and one of them is selected as the
master. What happens after that?

a. what will be the outcome of this transition SLAVE-->LEADER_ELECTION on
each participant ?
b. What will be the new idealstate which will allow one of the participants
to become MASTER and others SLAVE.

Another feature I thought about sometime back is conditional transition.
Basically have a transition that can have two outcomes, so in this case we
can have something like LEADER_ELECTION -> MASTER_READY, SLAVE and then do
the election in that transition and either go to MASTER_READY or go back to
SLAVE state. Helix can then promote MASTER_READY to MASTER. We might need
some changes in Helix but looks doable. We should file a jira for this
feature and  track this discussion .

Thanks for coming up with these ideas.

Kishore G

On Sat, Feb 23, 2013 at 7:22 PM, Puneet Zaroo <puneetzaroo@gmail.com> wrote:

> Kishore,
> Thanks for the detailed reply.
> Please see further comments inline.
> >
> > 3) Spectator is informed of the changes due to each state transition.
> >
> OK. Will that not cause a lot of overhead if there are a lot of
> Spectators in the system. Or was the rationale that there will be just
> a few spectators in the system.
> > 2) Yes it is possible to throttle the state transitions in a controlled
> > manner. You can basically specify the max number of transitions that can
> > occur at a resource, instance, instanceGroup, Cluster level. Helix will
> > ensure that none of those constraints are violated.
> >
> What I had in mind was throttling based on time and not the number of
> events. I.e. if a slave partition is lost, then the controller should
> wait for some configurable time before selecting another slave
> partition. This is to handle the case where a node is rebooting and we
> do not want its partitions to be moved to a new node immediately.
> > 1) Interpose Primary selection, yes it is possible  implement a custom
> > primary selection algorithm. Here is how we achieve that in LinkedIn
> >
> > a) A separate entity watches the ExternalView and as soon as it finds out
> > there is no primary for a partition, it can do the leader election and
> set
> > the idealstate. You can do this using the CustomCodeInvoker option which
> > ensures only one process watches the external view and computes the new
> > primary and sets the idealstate.
> >
> > Your suggestion of LEADER_ELECTION state sounds interesting. Can you
> > elaborate a bit more on the state machine ( states and transitions and
> > constraints). How will they get into this state?.
> >
> Are there any examples of how to use the CustomCodeInvoker ?
> Regarding a separate entity watching the ExternalView. Maybe I did not
> follow this fully, but the external entity looks similar to the
> controller; so I am not sure if this would solve the particular
> problem.
> We actually want the participants to take part in the decision of who
> should become the next Primary or Master. I havent thought this
> through completely, but one way could be to add a state
> "LEADER_ELECTION" between the states "SLAVE" and "MASTER". In the
> "LEADER_ELECTION" state the participants communicate with each other
> and decide who should be the next Master, and the participant elected
> as the next "Master" sets the IdealState.  This is fully auto mode,
> except for one transition "LEADER_ELECTION" -> "MASTER" which is
> custom.
> Perhaps there are simpler ways of doing this.
> thanks,
> - Puneet
> > On Thu, Feb 21, 2013 at 5:22 PM, Puneet Zaroo <puneetzaroo@gmail.com>
> wrote:
> >>
> >> I am a helix newbie. I have read the paper and the wiki pages and am
> >> just starting to get familiar with the source code. I had a few
> >> questions :
> >>
> >> 1) Is it possible to interpose on Primary selection. I.e. instead of
> >> relying completely on Helix to select a Primary, is it possible to
> >> implement a voting based protocol, where the replicas have a say in
> >> who becomes the next primary. One possible way would be to have a
> >> state "LEADER_ELECTION", in which the replicas do the voting, and
> >> finally just the winner sets the ideal state with itself as the
> >>
> >> Are there any gotchas in what I outlined above, or is there a
> >> completely alternative and better way of doing this ?
> >>
> >> 2) Is it possible to throttle state transitions. E.g. If a node goes
> >> offline, the replicas hosted on it should not be transferred to a new
> >> node immediately; but in a throttled manner.
> >>
> >> 3) When is a spectator informed of the new ExternalView ? Is it when
> >> currentState becomes equal to the idealState, or are they informed on
> >> all state changes due to each state transition.
> >>
> >> thanks,
> >> - Puneet
> >
> >

View raw message