helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "vlad.gm@gmail.com" <vlad...@gmail.com>
Subject Re: keeping the master node up during bootstrap
Date Tue, 08 Apr 2014 21:08:28 GMT
My modified code looks like:

/* Setup a Helix cluster for the KVStore */
public static void setupCluster() {
assert(cluster != null);
clusterSetup.addCluster(cluster, true);

        ConstraintItemBuilder constraintItemBuilder = new
ConstraintItemBuilder();

        constraintItemBuilder

.addConstraintAttribute(ConstraintAttribute.MESSAGE_TYPE.toString(),
"STATE_TRANSITION")

.addConstraintAttribute(ConstraintAttribute.PARTITION.toString(), ".*")

.addConstraintAttribute(ConstraintAttribute.CONSTRAINT_VALUE.toString(),
"1");

        clusterSetup.getClusterManagementTool().setConstraint(cluster,
                ClusterConstraints.ConstraintType.MESSAGE_CONSTRAINT,
                "constraint1", constraintItemBuilder.build());
    }

I will try to see whether it works in every situation.

Regards,
Vlad


On Tue, Apr 8, 2014 at 8:59 AM, Vlad Balan <vlad.gm@gmail.com> wrote:

> Hi Kishore,
>
> I managed to implement the bootstrapping using the constraint and it
> appears to be running as expected. I will post my code shortly.
>
> Regards,
> Vlad
>
> On Apr 8, 2014, at 8:27 AM, kishore g <g.kishore@gmail.com> wrote:
>
> Hi Vlad,
>
> Did you get a chance to play with the constraint.  I can write a sample
> code today to try this.
>
> Thanks,
> Kishore G
>
>
> On Thu, Apr 3, 2014 at 5:45 PM, vlad.gm@gmail.com <vlad.gm@gmail.com>wrote:
>
>>
>> Thank you Kanak and Kishore! I will try enforcing the per-partition
>> constraint and let you know if somehow it does not work. I was looking at
>> the throttling documentation, but somehow missed that a per-partition
>> constraint was an option!
>>
>> Regards,
>> Vlad
>>
>>
>> On Thu, Apr 3, 2014 at 5:42 PM, kishore g <g.kishore@gmail.com> wrote:
>>
>>> Hi Vlad,
>>>
>>> You can try setting the transition priority order and a constraint that
>>> there should be only one transition per partition across the cluster.
>>>
>>> So the transition priority could be something like
>>>
>>> Slave-Master
>>> Offfline -> Bootstrap
>>> Bootstrap->Slave
>>> Slave->Master
>>>
>>> For the rest not sure if order matters.
>>>
>>> Also set the max transitions constraint to 1 per partition.
>>>
>>> The reason I put Slave-Master before Offline->Bootstrap is to ensure
>>> that availability is given more importance. For example if you have 3
>>> nodes, N1, N2, N3. N1 is Master, N2 is Slave, and N3 is down. If N1 goes
>>> down and N3 comes up at the same time. We probably dont want to wait for N3
>>> to bootstrap before promoting N2 to Master.
>>>
>>> I haven't tested this but assuming the constraints enforcement works,
>>> this should do the trick.
>>>
>>> Does this make sense? Let me know if this does not work, we can add a
>>> test case.
>>>
>>> thanks,
>>> Kishore G
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Apr 3, 2014 at 4:57 PM, vlad.gm@gmail.com <vlad.gm@gmail.com>wrote:
>>>
>>>>
>>>> Dear all,
>>>>
>>>> I am trying to construct a state model with the following transition
>>>> diagram:
>>>>
>>>> OFFLINE -> BOOTSTRAPPING <---> SLAVE <-----> MASTER
>>>>          <-----------------------------------
>>>>
>>>> That is, an offline mode can go into a bootstraping state, from the
>>>> bootstrap state it can go into a slave state,
>>>> from slave it can go from master, from master to slave and from slave
>>>> it can go offline.
>>>>
>>>> Assume that if I have a partition with two nodes pf1 and pf2 and a
>>>> partition partition_0 with the following ideal state:
>>>>
>>>> partition_0: pf2: MASTER pf1: SLAVE,
>>>>
>>>> and that currently pf1 is serving as a master. When pf2 boots, Helix
>>>> will issue, almost simultaneously, two commands:
>>>> for pf1: transition from MASTER to SLAVE
>>>> for pf2: transition from BOOTSTRAPPING to SLAVE
>>>>
>>>> My understanding is that this happens since Helix is trying to execute
>>>> as many commands in parallel and since the last state
>>>> has pf2 as master. However, the transition from BOOTSTRAPPING to SLAVE
>>>> for pf2 involves a long data copy step, so
>>>> I would like to keep pf1 as a master in the meanwhile. I tried
>>>> prioritizing the transition from BOOTSTRAPPING to SLAVE
>>>> over the transition from MASTER to SLAVE, however Helix still issues
>>>> them in parallel (as it should).
>>>>
>>>> I was wondering what my options would be in order to keep the master up
>>>> while the future master is bootstrapping. Could
>>>> a throttling in the number of transitions be enforced at partition
>>>> level? Could I somehow specify that a state with a slave
>>>> and a bootstrapping node is undesirable?
>>>>
>>>> As a note, I have also looked at the RSync-replicateed filesystem
>>>> example. The reason for not using the OfflineOnline or the
>>>> MasterSlave model in my application is that I would like the
>>>> bootstrapping node to receive updates from clients, i.e. be visible
>>>> during the bootstrap. For this reason, I am introducing the new
>>>> BOOTSTRAPPING phase in-between OFFLINE and SLAVE.
>>>>
>>>> Regards,
>>>> Vlad
>>>>
>>>>
>>>> PS: The state model definition is as follows:
>>>>
>>>> builder.addState(MASTER, 1);
>>>>
>>>>
>>>>
>>>>             builder.addState(SLAVE, 2);
>>>>
>>>>
>>>>
>>>>             builder.addState(BOOTSTRAP, 3);
>>>>
>>>>
>>>>
>>>>             builder.addState(OFFLINE);
>>>>
>>>>
>>>>
>>>>             builder.addState(DROPPED);
>>>>
>>>>
>>>>
>>>>             // Set the initial state when the node starts
>>>>
>>>>
>>>>
>>>>             builder.initialState(OFFLINE);
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>             // Add transitions between the states.
>>>>
>>>>
>>>>
>>>>             builder.addTransition(OFFLINE, BOOTSTRAP, 4);
>>>>
>>>>
>>>>
>>>>             builder.addTransition(BOOTSTRAP, SLAVE, 5);
>>>>
>>>>
>>>>
>>>>             builder.addTransition(SLAVE, MASTER, 6);
>>>>
>>>>
>>>>
>>>>             builder.addTransition(MASTER, SLAVE, 3);
>>>>
>>>>
>>>>
>>>>             builder.addTransition(SLAVE, OFFLINE, 2);
>>>>
>>>>
>>>>
>>>>             builder.addTransition(OFFLINE, DROPPED, 1);
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>             // set constraints on states.
>>>>
>>>>
>>>>
>>>>             // static constraint
>>>>
>>>>
>>>>
>>>>             builder.upperBound(MASTER, 1);
>>>>
>>>>
>>>>
>>>>             // dynamic constraint, R means it should be derived based
>>>> on the replication
>>>>
>>>>
>>>>             // factor.
>>>>
>>>>
>>>>
>>>>             builder.dynamicUpperBound(SLAVE, "R");
>>>>
>>>
>>>
>>
>

Mime
View raw message