zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charith Wickramarachchi <charith.dhanus...@gmail.com>
Subject Re: Sequential Id generation with Zookeeper - Tips to improve performance
Date Wed, 13 Jun 2012 10:52:56 GMT
Thanks Sam for the reply.

The contention happens when we to setData from muliple clients right ? (due
to locking). But here even when i try using a single client the throughput
is in 20-30 id/s range.
Did you do any tuning in the server side. ?

regards,
Charith

On Wed, Jun 13, 2012 at 6:17 AM, Sam Tunnicliffe <sam@beobal.com> wrote:

> Hi Charith,
>
> can you partition your counters somehow, rather than having a single
> global counter? This would let you use multiple znodes, reducing
> contention during the setData call.
> We use almost exactly the same approach as you in our zookeeper backed
> sequence server[1] but with sharded counters and haven't had any
> issues with throughput (sorry, I don't have perf numbers to hand, but
> peak load pushes a few hundred r/s through)
>
> Cheers,
> Sam
>
> [1] https://github.com/talis/H1
>
> On 13 June 2012 05:20, Charith Wickramarachchi
> <charith.dhanushka@gmail.com> wrote:
> > Hi All ,
> >
> > I'm evaluating Apache Zookeepers to use in my project where i have a
> > requirement to generate unique and sequential message ids(long) across a
> > cluster.
> > After some  web search  i found[1] and some other different suggestions.
> I
> > started with the solution suggested in [1] but even with single client  i
> > only got a throughput around 20-30 ids/sec. But according to the
> performance
> > numbers pointed in the website i think i should be able get better
> results (
> > i m expecting to get at least 1000 id/s as i m using this for a MoM where
> > for each message i'll be calling zk id generator.)
> >
> > In this case I m using ZK version 3.3.4 running in my local machine with
> > default settings and client also running in the same machine. I m
> attaching
> > my id generator class.
> >
> > Any tips to improve the performance of this is highly appreciated. Am i
> > doing some thing wrong ?
> >
> >
> > [1]
> http://zookeeper-user.578899.n2.nabble.com/Unique-Id-Generation-td2688494.html
> >
> > cheers,
> > Charith
> > --
> > Charith Dhanushka Wickramarachchi
> > http://charithwiki.blogspot.com/
> >
>



-- 
Charith Dhanushka Wickramarachchi
http://charithwiki.blogspot.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message