zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: ZooKeeper performance
Date Mon, 03 Oct 2011 19:02:52 GMT
You can do that, but you don't have very many clients so sharding won't take
you very far.

But don't let me rain on your parade.  ZK is awesome and can be pushed to
pretty incredible lengths.

On Mon, Oct 3, 2011 at 11:29 AM, Dima Gutzeit
<dima.gutzeit@mailvision.com>wrote:

> Multiple zookeeper clusters with data partitioning (sharding) ?
>
> Regards,
> Dima Gutzeit.
>
> Sent from my iPhone
>
> On 3 באוק 2011, at 20:25, Ted Dunning <ted.dunning@gmail.com> wrote:
>
> > It definitely can be done, but what if your application grows by 10x?
> >
> > On Mon, Oct 3, 2011 at 10:56 AM, Dima Gutzeit
> > <dima.gutzeit@mailvision.com>wrote:
> >
> >> Very nice analysis at the link, thanks.
> >>
> >> Assuming my nodes are around the same size ~100 bytes, ensemble of 5
> >> servers and 10 clients can easily support the required ~20,000
> >> operations per second, no ?
> >>
> >> I will run a smoke-test to prove it ofcourse ...
> >>
> >> Regards,
> >> Dima Gutzeit.
> >>
> >> Sent from my iPhone
> >>
> >> On 3 באוק 2011, at 19:41, "Fournier, Camille F."
> >> <Camille.Fournier@gs.com> wrote:
> >>
> >>> It's pretty easy to set up a zk-smoketest to simulate what you are
> doing.
> >> We can't answer this question without knowing how big the data you're
> >> writing, etc etc. I would recommend testing it out yourself on realistic
> >> data sizes.
> >>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/ServiceLatencyOverview
> >>>
> >>> -----Original Message-----
> >>> From: Dima Gutzeit [mailto:dima.gutzeit@mailvision.com]
> >>> Sent: Monday, October 03, 2011 1:36 PM
> >>> To: user@zookeeper.apache.org
> >>> Subject: Re: ZooKeeper performance
> >>>
> >>> Inline.
> >>>
> >>> Regards,
> >>> Dima Gutzeit.
> >>>
> >>> Sent from my iPhone
> >>>
> >>> On 3 באוק 2011, at 19:31, Ted Dunning <ted.dunning@gmail.com>
wrote:
> >>>
> >>>> Questions in-line
> >>>>
> >>>> On Mon, Oct 3, 2011 at 10:19 AM, Dima Gutzeit
> >>>> <dima.gutzeit@mailvision.com>wrote:
> >>>>
> >>>>> Hi,
> >>>>>
> >>>>> I have some performance related question.
> >>>>>
> >>>>> I am running a cluster of 5 zookeeper machines, each one has dual
> quad
> >>>>> core Xeon 2.5 Ghz, 8 gb RAM.
> >>>>>
> >>>>> I want achieve the following numbers:
> >>>>>
> >>>>> 10 clients producing in total (at peaks):
> >>>>>
> >>>>
> >>>> Each producing this?
> >>>
> >>> Total.
> >>>>
> >>>>
> >>>>> 3K add nodes, 3K delete nodes and 10K watches. Per second.
> >>>>>
> >>>>
> >>>> What do you mean by 10K watches?  10,000 watch notifications?
> >>>
> >>> Yes.
> >>>>
> >>>> Does those numbers make any sense ?
> >>>>>
> >>>>
> >>>> This is a bit high.  It sounds like you are trying to use ZK as a
> >> message
> >>>> bus rather than a coordination service.  You can do this, but the
> >> throughput
> >>>> you can achieve is limited.  If you want higher throughput, it is
> better
> >> to
> >>>> have ZK coordinate a higher performance messaging system such as
> Kafka.
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message