zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Shraer <shra...@gmail.com>
Subject Re: Zookeeper in Kafka has gets a very low Performance
Date Thu, 13 Sep 2012 05:07:02 GMT
Maybe it has something to do with the frequent writes delaying reads ?
If a write is submitted to a follower and then a read (getData),
no matter if the write and the read are to the same data, the read
will block until the write completes. That's what ZOOKEEPER-1505 tries
to address.


Alex


On Wed, Sep 12, 2012 at 6:44 PM, sun liwei <sliveysun@gmail.com> wrote:
> Let me give more details:
> I have two zookeeper clusters: A(3.3.4) and B(3.4.3).
> A is used by Kafka only. B servers other application rather than Kafka.
> The performance of A is very bad no matter which path I read with getData;
> But B works well.
>
> My suspicion is that Kafka makes A slowing down. There are 1000+
> consumers(each consumer is a node) in A. Kafka updates the value of all
> these consumers very frequently. So, that means the write operations in A
> is much more than read operations.
>
>
> On Wed, Sep 12, 2012 at 6:33 PM, sun liwei <sliveysun@gmail.com> wrote:
>
>> The version of zookeeper is 3.3.4.
>>
>>
>> On Wed, Sep 12, 2012 at 6:07 PM, sun liwei <sliveysun@gmail.com> wrote:
>>
>>> Here is what I haveļ¼š
>>>
>>>
>>>    - zookeeper cluster with 3 severs
>>>    - 1700+ kafka topics (that means there are 1700+ children under path
>>>    '/brokers/topics'), these topics are written frequently by kafka.
>>>    - log file size is 67108880 byte,  snapshot file size is 4192072
>>>    byte, new log/snapshot files are created every one or two minutes
>>>
>>> The problem is that it takes more than 40ms to getData of a single topic.
>>> I have another zookeeper cluster and the getData takes no more than 2ms to
>>> getData of a single path. This cluster doesn't server kafka.
>>>
>>> Any body knows why the performance is so bad?
>>>
>>> Many thanks and best regards.
>>>
>>> Liwei
>>>
>>
>>

Mime
View raw message