helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kishore g <g.kish...@gmail.com>
Subject Re: A few questions about helix.
Date Fri, 22 Feb 2013 02:17:22 GMT
Thanks Puneet for the interesting questions. I will answer them in a
slightly different order since there are multiple answers to 1).


3) Spectator is informed of the changes due to each state transition.

2) Yes it is possible to throttle the state transitions in a controlled
manner. You can basically specify the max number of transitions that can
occur at a resource, instance, instanceGroup, Cluster level. Helix will
ensure that none of those constraints are violated.

1) Interpose Primary selection, yes it is possible  implement a custom
primary selection algorithm. Here is how we achieve that in LinkedIn

a) A separate entity watches the ExternalView and as soon as it finds out
there is no primary for a partition, it can do the leader election and set
the idealstate. You can do this using the CustomCodeInvoker option which
ensures only one process watches the external view and computes the new
primary and sets the idealstate.

Your suggestion of LEADER_ELECTION state sounds interesting. Can you
elaborate a bit more on the state machine ( states and transitions and
constraints). How will they get into this state?.


Thanks,
Kishore G


On Thu, Feb 21, 2013 at 5:22 PM, Puneet Zaroo <puneetzaroo@gmail.com> wrote:

> I am a helix newbie. I have read the paper and the wiki pages and am
> just starting to get familiar with the source code. I had a few
> questions :
>
> 1) Is it possible to interpose on Primary selection. I.e. instead of
> relying completely on Helix to select a Primary, is it possible to
> implement a voting based protocol, where the replicas have a say in
> who becomes the next primary. One possible way would be to have a
> state "LEADER_ELECTION", in which the replicas do the voting, and
> finally just the winner sets the ideal state with itself as the
> PRIMARY.
>
> Are there any gotchas in what I outlined above, or is there a
> completely alternative and better way of doing this ?
>
> 2) Is it possible to throttle state transitions. E.g. If a node goes
> offline, the replicas hosted on it should not be transferred to a new
> node immediately; but in a throttled manner.
>
> 3) When is a spectator informed of the new ExternalView ? Is it when
> currentState becomes equal to the idealState, or are they informed on
> all state changes due to each state transition.
>
> thanks,
> - Puneet
>

Mime
View raw message