zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dima Gutzeit <dima.gutz...@mailvision.com>
Subject Re: ZooKeeper performance
Date Mon, 03 Oct 2011 18:29:27 GMT
Multiple zookeeper clusters with data partitioning (sharding) ?

Regards,
Dima Gutzeit.

Sent from my iPhone

On 3 באוק 2011, at 20:25, Ted Dunning <ted.dunning@gmail.com> wrote:

> It definitely can be done, but what if your application grows by 10x?
>
> On Mon, Oct 3, 2011 at 10:56 AM, Dima Gutzeit
> <dima.gutzeit@mailvision.com>wrote:
>
>> Very nice analysis at the link, thanks.
>>
>> Assuming my nodes are around the same size ~100 bytes, ensemble of 5
>> servers and 10 clients can easily support the required ~20,000
>> operations per second, no ?
>>
>> I will run a smoke-test to prove it ofcourse ...
>>
>> Regards,
>> Dima Gutzeit.
>>
>> Sent from my iPhone
>>
>> On 3 באוק 2011, at 19:41, "Fournier, Camille F."
>> <Camille.Fournier@gs.com> wrote:
>>
>>> It's pretty easy to set up a zk-smoketest to simulate what you are doing.
>> We can't answer this question without knowing how big the data you're
>> writing, etc etc. I would recommend testing it out yourself on realistic
>> data sizes.
>>>
>>>
>> https://cwiki.apache.org/confluence/display/ZOOKEEPER/ServiceLatencyOverview
>>>
>>> -----Original Message-----
>>> From: Dima Gutzeit [mailto:dima.gutzeit@mailvision.com]
>>> Sent: Monday, October 03, 2011 1:36 PM
>>> To: user@zookeeper.apache.org
>>> Subject: Re: ZooKeeper performance
>>>
>>> Inline.
>>>
>>> Regards,
>>> Dima Gutzeit.
>>>
>>> Sent from my iPhone
>>>
>>> On 3 באוק 2011, at 19:31, Ted Dunning <ted.dunning@gmail.com> wrote:
>>>
>>>> Questions in-line
>>>>
>>>> On Mon, Oct 3, 2011 at 10:19 AM, Dima Gutzeit
>>>> <dima.gutzeit@mailvision.com>wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I have some performance related question.
>>>>>
>>>>> I am running a cluster of 5 zookeeper machines, each one has dual quad
>>>>> core Xeon 2.5 Ghz, 8 gb RAM.
>>>>>
>>>>> I want achieve the following numbers:
>>>>>
>>>>> 10 clients producing in total (at peaks):
>>>>>
>>>>
>>>> Each producing this?
>>>
>>> Total.
>>>>
>>>>
>>>>> 3K add nodes, 3K delete nodes and 10K watches. Per second.
>>>>>
>>>>
>>>> What do you mean by 10K watches?  10,000 watch notifications?
>>>
>>> Yes.
>>>>
>>>> Does those numbers make any sense ?
>>>>>
>>>>
>>>> This is a bit high.  It sounds like you are trying to use ZK as a
>> message
>>>> bus rather than a coordination service.  You can do this, but the
>> throughput
>>>> you can achieve is limited.  If you want higher throughput, it is better
>> to
>>>> have ZK coordinate a higher performance messaging system such as Kafka.
>>

Mime
View raw message