zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Camille Fournier <skami...@gmail.com>
Subject Re: Client connections keep growing and growing
Date Sun, 16 Oct 2011 15:58:48 GMT
Seems like a hbase problem to me... Zk doesn't randomly create or hang on to
client connections. Did you ask on the hbase lists?

C

On Sunday, October 16, 2011, Ronen Itkin <ronen@taykey.com> wrote:
> Hey all!
> How are you?
>
> I am currently using Zookeeper 3.3.3-cdh3u1, installed as a distributed
> cluster with 3 nodes.
> Though I have installed Zookeeper separately, its only purpose right now
is
> to serve HBase clients (hbase-0.90.3-cdh3u1).
> Seems like everything is working great for a day or two and that out the
> blue, Zookeeper refuses to serve clients requests,
> alerting for the wide known issue of exceeding the defined maximum
> connections allowed for a client.
> I changed this value to 500 and it seems that it just held on for more
time
> and finally reached the threshold of 500 and then stopped working.
> So, I set it to 1000 and right now I am waiting to see what will happen.
>
> Meanwhile, by auditing the Zookeeper connections manually it seems like
they
> keep growing in a slow - promising pace towards the threshold of 1000
> connections.
>
> Can someone think of a reason why it happens? Why it keep growing? Does
> someone has any other suggestions regarding this issue?
> What is the recommended value for Zookeeper maximum connections per
client?
> can it overload the server if set to a high value? what is  a high value
in
> this case?
>
> If it helps, this is my environment design:
>
>   - Server 1 - Hadoop NameNode, JobTracker, *Hbase Master (Including
Rest)*,
>   *Zookeeper01*
>   - Server 2 - Hadoop SecondaryMaster, HBase *Backup Master (Including
>   Rest)*, *Zookeeper02*
>   - Server 3 - *Zookeeper03*
>   - Servers 4-6 - Hadoop DataNode,TaskTracker, *HBase Region Server.*
>
> The type of operations I am committing on HBase using Zookeeper are:
>
>   - Simple read/writes/updates using SCAN / MapReduce / Cascading ..
>   - Queries of the Rest service.
>
>
> Thanks a lot !!!
> *   *Ronen Itkin
>
> <http://www.taykey.com/>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message