zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sun liwei <slivey...@gmail.com>
Subject Re: Zookeeper in Kafka has gets a very low Performance
Date Thu, 13 Sep 2012 01:44:22 GMT
Let me give more details:
I have two zookeeper clusters: A(3.3.4) and B(3.4.3).
A is used by Kafka only. B servers other application rather than Kafka.
The performance of A is very bad no matter which path I read with getData;
But B works well.

My suspicion is that Kafka makes A slowing down. There are 1000+
consumers(each consumer is a node) in A. Kafka updates the value of all
these consumers very frequently. So, that means the write operations in A
is much more than read operations.

On Wed, Sep 12, 2012 at 6:33 PM, sun liwei <sliveysun@gmail.com> wrote:

> The version of zookeeper is 3.3.4.
> On Wed, Sep 12, 2012 at 6:07 PM, sun liwei <sliveysun@gmail.com> wrote:
>> Here is what I haveļ¼š
>>    - zookeeper cluster with 3 severs
>>    - 1700+ kafka topics (that means there are 1700+ children under path
>>    '/brokers/topics'), these topics are written frequently by kafka.
>>    - log file size is 67108880 byte,  snapshot file size is 4192072
>>    byte, new log/snapshot files are created every one or two minutes
>> The problem is that it takes more than 40ms to getData of a single topic.
>> I have another zookeeper cluster and the getData takes no more than 2ms to
>> getData of a single path. This cluster doesn't server kafka.
>> Any body knows why the performance is so bad?
>> Many thanks and best regards.
>> Liwei

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message