zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ronen Itkin <ro...@taykey.com>
Subject Client connections keep growing and growing
Date Sun, 16 Oct 2011 11:52:19 GMT
Hey all!
How are you?

I am currently using Zookeeper 3.3.3-cdh3u1, installed as a distributed
cluster with 3 nodes.
Though I have installed Zookeeper separately, its only purpose right now is
to serve HBase clients (hbase-0.90.3-cdh3u1).
Seems like everything is working great for a day or two and that out the
blue, Zookeeper refuses to serve clients requests,
alerting for the wide known issue of exceeding the defined maximum
connections allowed for a client.
I changed this value to 500 and it seems that it just held on for more time
and finally reached the threshold of 500 and then stopped working.
So, I set it to 1000 and right now I am waiting to see what will happen.

Meanwhile, by auditing the Zookeeper connections manually it seems like they
keep growing in a slow - promising pace towards the threshold of 1000
connections.

Can someone think of a reason why it happens? Why it keep growing? Does
someone has any other suggestions regarding this issue?
What is the recommended value for Zookeeper maximum connections per client?
can it overload the server if set to a high value? what is  a high value in
this case?

If it helps, this is my environment design:

   - Server 1 - Hadoop NameNode, JobTracker, *Hbase Master (Including Rest)*,
   *Zookeeper01*
   - Server 2 - Hadoop SecondaryMaster, HBase *Backup Master (Including
   Rest)*, *Zookeeper02*
   - Server 3 - *Zookeeper03*
   - Servers 4-6 - Hadoop DataNode,TaskTracker, *HBase Region Server.*

The type of operations I am committing on HBase using Zookeeper are:

   - Simple read/writes/updates using SCAN / MapReduce / Cascading ..
   - Queries of the Rest service.


Thanks a lot !!!
*   *Ronen Itkin

<http://www.taykey.com/>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message