hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Logvinov <monkego...@yandex.ru>
Subject Multiple connections to ZooKeeper are being constantly created and disposed
Date Thu, 29 Jan 2015 13:48:56 GMT
Hello,

I'm seeing an issue similar to the one described here: http://webmail.dev411.com/t/hbase/user/149jddw9gh/hbase-establishes-session-with-zookeeper-and-close-the-session-immediately/.
The only difference is that I don't have a replication set up since HBase is running in a
single-node mode on my local machine. Env is:
- hbase-0.98.8-hadoop2
- embedded ZK managed by HBase
- Titan DB running atop of HBase

The problem is that as I can see from logs, connection establishment and disposal happens
many-many times over the time program is running which dramatically slows down operation of
the program itself. I ran debugger several times and most of the calls seem to originate from
HBaseAdmin#tableExists(TableName) method, where CatalogTracker is cleaned up which in turn
closes ZK connection.

Is this the way it is supposed to work? Can anything be done to establish ZK connection, keep
it an reuse whenever necessary?

Here's a snippet of log (I tried to select relevant info from my point of view cause there's
a lot of it):

[2015-01-29_14:43:22.701] [pool-195-thread-1] TRACE o.a.hadoop.hbase.zookeeper.ZKUtil - hconnection-0x5849c845
opening connection to ZooKeeper ensemble=127.0.0.1:2181
[2015-01-29_14:43:22.701] [pool-195-thread-1] INFO  o.a.h.h.z.RecoverableZooKeeper - Process
identifier=hconnection-0x5849c845 connecting to ZooKeeper ensemble=127.0.0.1:2181
[2015-01-29_14:43:22.701] [pool-195-thread-1] INFO  org.apache.zookeeper.ZooKeeper - Initiating
client connection, connectString=127.0.0.1:2181 sessionTimeout=90000 watcher=hconnection-0x5849c845,
quorum=127.0.0.1:2181, baseZNode=/hbase
...
[2015-01-29_14:43:22.701] [pool-195-thread-1-SendThread(127.0.0.1:2181)] INFO  org.apache.zookeeper.ClientCnxn
- Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate
using SASL (unknown error)
...
[2015-01-29_14:43:22.702] [pool-195-thread-1-SendThread(127.0.0.1:2181)] INFO  org.apache.zookeeper.ClientCnxn
- Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session
...
[2015-01-29_14:43:22.703] [pool-195-thread-1-SendThread(127.0.0.1:2181)] INFO  org.apache.zookeeper.ClientCnxn
- Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x14b35473ea3036d,
negotiated timeout = 40000
...
[2015-01-29_14:43:22.704] [pool-195-thread-1-EventThread] DEBUG o.a.h.h.zookeeper.ZooKeeperWatcher
- hconnection-0x5849c845-0x14b35473ea3036d connected
...
[2015-01-29_14:43:22.705] [pool-195-thread-1] TRACE o.a.hadoop.hbase.zookeeper.ZKUtil - catalogtracker-on-hconnection-0x5849c845
opening connection to ZooKeeper ensemble=127.0.0.1:2181
[2015-01-29_14:43:22.705] [pool-195-thread-1] INFO  o.a.h.h.z.RecoverableZooKeeper - Process
identifier=catalogtracker-on-hconnection-0x5849c845 connecting to ZooKeeper ensemble=127.0.0.1:2181
[2015-01-29_14:43:22.705] [pool-195-thread-1] INFO  org.apache.zookeeper.ZooKeeper - Initiating
client connection, connectString=127.0.0.1:2181 sessionTimeout=90000 watcher=catalogtracker-on-hconnection-0x5849c845,
quorum=127.0.0.1:2181, baseZNode=/hbase
...
[2015-01-29_14:43:22.708] [pool-195-thread-1-SendThread(127.0.0.1:2181)] INFO  org.apache.zookeeper.ClientCnxn
- Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x14b35473ea3036e,
negotiated timeout = 40000
...
[2015-01-29_14:43:22.713] [pool-195-thread-1] TRACE o.a.h.hbase.client.ClientScanner - Scan
table=hbase:meta, startRow=titan,,
...
[2015-01-29_14:43:22.717] [pool-195-thread-1] TRACE o.a.h.hbase.client.ClientScanner - Finished
{ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}
...
[2015-01-29_14:43:22.717] [pool-195-thread-1] DEBUG o.a.h.hbase.catalog.CatalogTracker - Stopping
catalog tracker org.apache.hadoop.hbase.catalog.CatalogTracker@48c9ae7f
[2015-01-29_14:43:22.717] [pool-195-thread-1] INFO  o.a.h.h.c.HConnectionManager$HConnectionImplementation
- Closing zookeeper sessionid=0x14b35473ea3036d
...
[2015-01-29_14:43:22.821] [pool-195-thread-1] DEBUG org.apache.zookeeper.ZooKeeper - Closing
session: 0x14b35473ea3036e
[2015-01-29_14:43:22.821] [pool-195-thread-1] DEBUG org.apache.zookeeper.ClientCnxn - Closing
client for session: 0x14b35473ea3036e

-- 
Best regards,
Andrew

Mime
View raw message