zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jmmec <jmmec2...@gmail.com>
Subject Re: ZK 3.4.5: Very Strange Write Latency Problem?
Date Fri, 21 Feb 2014 13:51:44 GMT
Thanks Camille, I definitely understand!  :)

The two questions at the top of mind regarding ZooKeeper are:
1. How does it calculate latencies?  I can dig into its code to see.
2. Is there anything in particular that might cause it to have the spiky
latency I've experienced?  I think I ruled out the snapshot behavior by
having a high snapCount.

Some other things I am planning to explore:
1. My test software is rightfully suspect, so I'll review it carefully
again and will simplify it further so that it is doing the absolute bare
minimum.
2. I'm running OpenJDK 1.7.0_60-ea so might swap to an earlier and/or
different distribution.
3. I'm running ZooKeeper 3.4.5 and might fall back to the 3.3.6 release.

Hopefully one of the items above will reveal the root cause.  Any other
suggestions are welcome.



On Thu, Feb 20, 2014 at 7:57 PM, Camille Fournier <camille@apache.org>wrote:

> I might suggest that you create a personal github and mock up a replication
> there :) I understand employers that own your code but unless someone knows
> the answer off the top of their head, odds of finding the cause are low
> without something that replicates it, and knowing how busy most of us are
> here I don't know that we'll have time to do that for you.
>
> C
>
>
> On Thu, Feb 20, 2014 at 9:41 PM, jmmec <jmmec2009@gmail.com> wrote:
>
> > Thanks again,
> >
> > Unfortunately I can't share the test code since it is technically the
> > property of my employer.
> >
> > It's very strange behavior.  I think I've said that several times now.
> > ha...
> >
> > Appreciate any additional help or advice or suggestions from everyone and
> > anyone and their brother or sister.
> >
> >
> >
> > On Thu, Feb 20, 2014 at 8:10 PM, Camille Fournier <camille@apache.org
> > >wrote:
> >
> > > Can you share the test code somewhere (github maybe?)?
> > >
> > > Thanks,
> > > C
> > >
> > >
> > > On Thu, Feb 20, 2014 at 9:08 PM, jmmec <jmmec2009@gmail.com> wrote:
> > >
> > > > Thanks for the quick reply.
> > > >
> > > > I did not try the "slow" test using a normal disk drive, however I
> > first
> > > > discovered this problem when writing to a 7200RPM disk drive at a
> much
> > > > higher messaging rate (e.g. 1500 to 3000 creates/sec rather than 84
> > > > creates/sec).  This is what caused me to start simplifying the
> > > > configuration trying to find the root cause.  As part of that
> > > > investigation, I created a RAM disk to avoid the hard drive, but the
> > hard
> > > > drive wasn't the problem.  I just haven't switched back to the hard
> > > drive.
> > > >
> > > > I don't know what ZooKeeper is doing internally, or how & why it is
> > > > deriving 76ms MAX latency.  The very regular periodic pattern
> suggests
> > > > something odd.
> > > >
> > > > Hmmmm.....
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message