helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kishore g <g.kish...@gmail.com>
Subject Re: State transitions of partitions
Date Wed, 27 Feb 2013 18:53:27 GMT
Hi Vinayak,

By default, a transition is not time bound(it can be a short one or really
long), you can do the data movement as part of the transition and return
from the transition after its complete.

Helix is aware of the pending transition and its maintained in ZK and is
modelled as a first class citizen of the state machine. It allows you to
specify throttling requirements on each transition type. The constraint can
be set per partition, resource, node or cluster.

Lets say you have some transition called STARTED-BOOTSTRAPPED where you
copy the data. And say you have 100 nodes in the cluster that needs to go
from STARTED-BOOTSTRAPED. You can simply set the end state of system like
N1:BOOTSTRAPPED
N2:BOOTSTRAPPED
...
....
N100:BOOTSTRAPED

and set a constraint that only 10 nodes should do this transition at once.

Helix will then ensure that only 10 nodes in the cluster are doing this
transition at a time. Note this is not like 10 batches of 10 at a time,
instead 10 at all times which means as soon as one transition is done it
will add another transition.

See TestConstraint on how to use these features, we probably have java
api's but no CLI ( Jason please correct me if I am wrong).

To see how its implemented see MessageThrottleStage

thanks,
Kishore G
















On Wed, Feb 27, 2013 at 10:16 AM, Vinayak Borkar <vborky@yahoo.com> wrote:

> Hi Guys,
>
>
> I am trying to understand how state transitions work in Helix. My
> understanding is that once a controller decides to perform a state
> transition, this information is conveyed to the relevant participants. A
> method corresponding to the transitions is invoked on the state model
> object at the participant corresponding to the partition whose state needs
> to change.
>
> When state transitions involve data movement, performing the actual
> transition at the participant is not an instantaneous activity. So while
> the invocation of the transition function triggers the actual data
> movement, is there a way for the participant to indicate that the
> transition is complete? What is the state of the system while a transition
> is being effected on the cluster?
>
> I am guessing Helix needs to model the state of an in-progress transition
> to correctly provide throttling guarantees on the cluster.
>
> Where should I be looking in the code to understand how this works?
>
>
> Thanks,
> Vinayak
>

Mime
View raw message