helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kishore g <g.kish...@gmail.com>
Subject Re: Prevent failback to MASTER after failover
Date Sun, 31 Mar 2013 05:56:05 GMT
No you dont have to change the state model to achieve this.

Intead of AUTO, AUTO_REBALANCE should work in your case. Simply change the
ideal state to look like this.

    "id": "Cluster",
        "NUM_PARTITIONS": "1",
        "REPLICAS": "2",
        "Partition_0" : [ ]

You can also achieve this in AUTO mode: when a node becomes master for a
partition, as part of the transition change the preference list in the
idealstate. So in this case change "Partition_0" : [ "node_1", "node_2" ]
to "Partition_0" : [ "node_2", "node_1" ] when node_2 becomes master.

In the next release, you will be able to add custom rebalancer code which
will allow you to make this change in the controller easily.

Kishore G

On Sat, Mar 30, 2013 at 10:21 PM, Ming Fang <mingfang@mac.com> wrote:

> Hi Kishore
> Our system requires deterministic placement of the MASTER and SLAVE.
> This is a sample of the idealstate file we're using
> {
>     "id": "Cluster",
>     "simpleFields":{
>         "IDEAL_STATE_MODE":"AUTO",
>         "NUM_PARTITIONS": "1",
>         "REPLICAS": "2",
>         "STATE_MODEL_DEF_REF":"MasterSlave"
>     },
>     "mapFields":{
>     },
>     "listFields":{
>         "Partition_0" : [ "node_1", "node_2" ]
>     }
> }
> In this example, node_1 is the MASTER.
> If node_1 dies then node_2 will take over.
> But if node_1 then get restarted, it will try to become MASTER again.
> We normally keep the died node down to avoid this problem.
> But I was hoping for a more elegant solution.
> One solution would be for node_1 to come up and realizes that node_2 has
> taken over due to the previous failure.
> In that case node_1 will decide to remain as a SLAVE node instead.
> Should this be done by the Controller instead?
> Should I create a new statemodel other than MASTER/SLAVE?
> On Mar 31, 2013, at 12:50 AM, kishore g <g.kishore@gmail.com> wrote:
> Hi MIng,
> There are couple of ways you can achieve that. Before providing an answer,
> how many partitions do you have. Did you generate the idealstate yourself
> or used Helix to come up with initial idealstate?
> The reason old master tries to become a master again is to distribute the
> load among the nodes currently alive. Otherwise the old node that comes
> back will never become a master for any partition and will remain idle
> until another failure happens in the system.
> thanks,
> Kishore G
> On Sat, Mar 30, 2013 at 8:01 PM, Ming Fang <mingfang@mac.com> wrote:
>> We're using MASTER SLAVE in AUTO model.
>> When the MASTER is killed, the failover is working properly as the SLAVE
>> transitions to become MASTER.
>> However if the failed MASTER is restarted, it will try to become MASTER
>> again.
>> This is causing a problem in our business logic.
>> Is there a way to prevent the failed instance from becoming MASTER again?

View raw message