zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Han <h...@cloudera.com>
Subject Re: Observers taking long time to serve requests
Date Wed, 17 May 2017 23:50:34 GMT
Observer server log should contain INFO level logging on whether it's
getting a diff or getting a snapshot from leader. To estimate how long the
data transfer took, look for "UPTODATE" INFO level log on observer. This is
the message logged after data sync between leader and observer finished.

On Wed, May 17, 2017 at 3:59 PM, rammohan ganapavarapu <
rammohanganap@gmail.com> wrote:

> Michael,
>
> Thanks for your suggestions, so do you know how to find when it is taking
> diff or full snapshot? in my case it looks like taking full snapshot. so
> from observing leader to snapshotting is taking 30mins.
>
> Ram
>
> On May 16, 2017 2:07 PM, "Michael Han" <hanm@cloudera.com> wrote:
>
> > When an observer (and in general a follower) restarted, it will go
> through
> > these stages:
> >
> > * Look for leader by starting a new leader election round and usually
> this
> > is quick as there is already a leader.
> > * Register with leader and begin synchronize phase - depends on the
> > observer state the sync might end up with DIFF (quicker) or SNAPSHOT
> > (longer).
> > * After sync phase observer will start serving traffic.
> >
> > Check the observer server log and record how long it took for each stage
> is
> > a good starting point..
> >
> >
> > On Tue, May 16, 2017 at 1:04 PM, Ben Sherman <bensherman@gmail.com>
> wrote:
> >
> > > With latency that high, I'd expect the thruput to be low, is 30m in
> line
> > > with what it takes to transfer a gig of data across that pipe?
> > >
> > > On Mon, May 15, 2017 at 4:19 PM, rammohan ganapavarapu <
> > > rammohanganap@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > We have a zookeeper cluster in multi region, we have leader in US
> EAST
> > > and
> > > > observers in US WEST, our data size is 1.3GB,network latency between
> > > leader
> > > > and observers is around 250ms, upon observer restart it is taking
> like
> > > > close to 30m for observer to serve traffic. Not sure why it's taking
> so
> > > > long to serve request. Does any one see such issue? any idea why it's
> > > > taking time?
> > > >
> > > > Thanks,
> > > > Ram
> > > >
> > >
> >
> >
> >
> > --
> > Cheers
> > Michael.
> >
>



-- 
Cheers
Michael.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message