Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@zookeeper.apache.org
Received-SPF: pass (nike.apache.org: domain of ralph.tice@gmail.com designates
 209.85.217.177 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <21EEE329-6E78-4FD6-BE34-189E93DF23EB@codepuppy.com>
References: <1065A2D3-E37E-4C9A-A561-BB8369AC48AC@codepuppy.com>
	<CANcXBFOhAbA1_ToyiuZAAG2AQj91imz+2EDN8Taz9zC_mYCR-g@mail.gmail.com>
	<87h9zcxb1h.fsf@ip-10-56-193-148.eu-west-1.compute.internal>
	<21EEE329-6E78-4FD6-BE34-189E93DF23EB@codepuppy.com>
Date: Sat, 11 Oct 2014 13:09:59 -0500
Message-ID: 
 <CAORF7jmdedqm6S51_4KN3fz0LVXTLTxnZexbQvefRKC+RUT_jw@mail.gmail.com>
Subject: Re: Changing leader to follower?
From: ralph tice <ralph.tice@gmail.com>
To: user@zookeeper.apache.org
Content-Type: multipart/alternative; boundary=089e01493a14e5888d05052992f1

--089e01493a14e5888d05052992f1
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

I'm not an expert but I don't think there is a magic bullet here, leader
election has to happen in this circumstance and that takes time.

You may be better served by building better resilience to eliminate
ZooKeeper's uptime from being a single point of failure in your services
layer.  Pinterest and Airbnb both have some prior art here,
http://engineering.pinterest.com/post/77933733851/zookeeper-resilience-at-p=
interest
and http://nerds.airbnb.com/smartstack-service-discovery-cloud/

I'm curious why you chose a cross-DC ensemble versus localized same-region
ensembles.  Don't you deal with a significant frequency of leader elections
from being in 3 regions anyway?


On Sat, Oct 11, 2014 at 11:21 AM, Jeff Potter <
jpotter-zookeeper@codepuppy.com> wrote:

>
> The reason I ask is that we=E2=80=99ve noticed, when running zookeeper cr=
oss-DC,
> that restarting the node that=E2=80=99s currently the leader causes a bri=
ef but
> real service interruption for 3 to 5 seconds while the rest of the cluste=
r
> elects a new leader and syncs. We=E2=80=99re on AWS, with 2 ZK nodes in U=
S-East, 2
> in US-West-2, and 1 in US-West (as a tie-breaker).
>
> It would seem taking a leader to follower status would be useful; and
> doing so without it actually being a stop / disconnect on all clients
> connect to the node. (Especially for doing rolling restarts of all nodes,
> e.g. XEN-108 bug.)
>
> -Jeff
>
>
>
> On Oct 10, 2014, at 10:16 AM, Ivan Kelly <ivank@apache.org> wrote:
>
> > Or just pause the process until someone else takes over.
> >
> > 1. kill -STOP <zookeeper_pid>
> > 2. // wait for election to happen
> > 3. kill -CONT <zookeeper_pid>
> >
> > This wont top it from becoming leader again. Also, client may migrate t=
o
> > other servers.
> >
> > -Ivan
> >
> > Alexander Shraer writes:
> >
> >> Hi,
> >>
> >> I don't think there's a direct way, although this seems a useful thing
> to
> >> add.
> >>
> >> One think you could do is to issue a reconfig changing the leader's
> >> leading/quorum port (through which
> >> it talks with the followers). This will cause it to give up leadership
> >> while keeping it in the cluster.
> >>
> >> Cheers,
> >> Alex
> >>
> >> On Fri, Oct 10, 2014 at 5:57 AM, Jeff Potter <
> >> jpotter-zookeeper@codepuppy.com> wrote:
> >>
> >>>
> >>> Hi,
> >>>
> >>> Is there a way to =E2=80=9Cretire=E2=80=9D a leader while keeping it =
in the cluster?
> >>>
> >>> Thanks,
> >>> Jeff
>
>

--089e01493a14e5888d05052992f1--