hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: HConnectionManager leaks with zookeeper conection oo many connections from /my.tomcat.server.com - max is 60
Date Fri, 12 Dec 2014 19:28:21 GMT
I cannot reproduce. I stood up a cdh5.2 server and then copy/pasted your
code adding in a put for each cycle.  I ran loop 1000 times and no
complaint from zk.

Tell me more (Is servlet doing single-threaded model?  A single
Configuration is being used or new ones are being created per servlet
invocation?

Below is code and output.

(For better perf, cache the connection)
St.Ack


  1 package org.apache.hadoop.hbase;
  2
  3 import java.io.IOException;
  4
  5 import org.apache.hadoop.conf.Configuration;
  6 import org.apache.hadoop.hbase.client.HConnection;
  7 import org.apache.hadoop.hbase.client.HConnectionManager;
  8 import org.apache.hadoop.hbase.client.HTableInterface;
  9 import org.apache.hadoop.hbase.client.Put;
 10 import org.apache.hadoop.hbase.util.Bytes;
 11
 12 public class TestConnnection {
 13   public static void main(String[] args) throws IOException {
 14     Configuration config = HBaseConfiguration.create();
 15     for (int i = 0; i < 1000; i++) {
 16       HConnection connection =
HConnectionManager.createConnection(config);
 17       HTableInterface table =
connection.getTable(TableName.valueOf("table1"));
 18       byte [] cf = Bytes.toBytes("t");
 19       try {
 20         byte [] bytes = Bytes.toBytes(i);
 21         Put p = new Put(bytes);
 22         p.add(cf, cf, bytes);
 23         table.put(p);
 24       } finally {
 25         table.close();
 26         connection.close();
 27       }
 28       System.out.println("i=" + i);
 29     }
 30   }
 31 }


....
2014-12-12 11:26:10,397 INFO  [main] zookeeper.RecoverableZooKeeper:
Process identifier=hconnection-0x70dfa475 connecting to ZooKeeper
ensemble=localhost:2181
2014-12-12 11:26:10,397 INFO  [main] zookeeper.ZooKeeper: Initiating client
connection, connectString=localhost:2181 sessionTimeout=90000
watcher=hconnection-0x70dfa475, quorum=localhost:2181, baseZNode=/hbase
2014-12-12 11:26:10,398 INFO  [main-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Opening socket connection to server localhost/
127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2014-12-12 11:26:10,398 INFO  [main-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Socket connection established to localhost/
127.0.0.1:2181, initiating session
2014-12-12 11:26:10,401 INFO  [main-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Session establishment complete on server localhost/
127.0.0.1:2181, sessionid = 0x14a3fe0664b083a, negotiated timeout = 40000
2014-12-12 11:26:10,405 DEBUG [main] client.ClientSmallScanner: Finished
with small scan at {ENCODED => 1588230740, NAME => 'hbase:meta,,1',
STARTKEY => '', ENDKEY => ''}
2014-12-12 11:26:10,406 INFO  [main]
client.HConnectionManager$HConnectionImplementation: Closing zookeeper
sessionid=0x14a3fe0664b083a
2014-12-12 11:26:10,407 INFO  [main] zookeeper.ZooKeeper: Session:
0x14a3fe0664b083a closed
2014-12-12 11:26:10,407 INFO  [main-EventThread] zookeeper.ClientCnxn:
EventThread shut down
i=997
2014-12-12 11:26:10,511 INFO  [main] zookeeper.RecoverableZooKeeper:
Process identifier=hconnection-0x22a7106 connecting to ZooKeeper
ensemble=localhost:2181
2014-12-12 11:26:10,511 INFO  [main] zookeeper.ZooKeeper: Initiating client
connection, connectString=localhost:2181 sessionTimeout=90000
watcher=hconnection-0x22a7106, quorum=localhost:2181, baseZNode=/hbase
2014-12-12 11:26:10,512 INFO  [main-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Opening socket connection to server
localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL
(unknown error)
2014-12-12 11:26:10,512 INFO  [main-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Socket connection established to
localhost/0:0:0:0:0:0:0:1:2181, initiating session
2014-12-12 11:26:10,515 INFO  [main-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Session establishment complete on server
localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x14a3fe0664b083b, negotiated
timeout = 40000
2014-12-12 11:26:10,519 DEBUG [main] client.ClientSmallScanner: Finished
with small scan at {ENCODED => 1588230740, NAME => 'hbase:meta,,1',
STARTKEY => '', ENDKEY => ''}
2014-12-12 11:26:10,521 INFO  [main]
client.HConnectionManager$HConnectionImplementation: Closing zookeeper
sessionid=0x14a3fe0664b083b
2014-12-12 11:26:10,521 INFO  [main] zookeeper.ZooKeeper: Session:
0x14a3fe0664b083b closed
2014-12-12 11:26:10,521 INFO  [main-EventThread] zookeeper.ClientCnxn:
EventThread shut down
i=998
2014-12-12 11:26:10,627 INFO  [main] zookeeper.RecoverableZooKeeper:
Process identifier=hconnection-0x3822f407 connecting to ZooKeeper
ensemble=localhost:2181
2014-12-12 11:26:10,627 INFO  [main] zookeeper.ZooKeeper: Initiating client
connection, connectString=localhost:2181 sessionTimeout=90000
watcher=hconnection-0x3822f407, quorum=localhost:2181, baseZNode=/hbase
2014-12-12 11:26:10,628 INFO  [main-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Opening socket connection to server localhost/
127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2014-12-12 11:26:10,629 INFO  [main-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Socket connection established to localhost/
127.0.0.1:2181, initiating session
2014-12-12 11:26:10,631 INFO  [main-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Session establishment complete on server localhost/
127.0.0.1:2181, sessionid = 0x14a3fe0664b083c, negotiated timeout = 40000
2014-12-12 11:26:10,637 DEBUG [main] client.ClientSmallScanner: Finished
with small scan at {ENCODED => 1588230740, NAME => 'hbase:meta,,1',
STARTKEY => '', ENDKEY => ''}
2014-12-12 11:26:10,638 INFO  [main]
client.HConnectionManager$HConnectionImplementation: Closing zookeeper
sessionid=0x14a3fe0664b083c
2014-12-12 11:26:10,639 INFO  [main] zookeeper.ZooKeeper: Session:
0x14a3fe0664b083c closed
2014-12-12 11:26:10,639 INFO  [main-EventThread] zookeeper.ClientCnxn:
EventThread shut down
i=999




On Fri, Dec 12, 2014 at 10:22 AM, Serega Sheypak <serega.sheypak@gmail.com>
wrote:

> Hi, I'm using CDH 5.2, 0.98
> I don't know how to use it correctly. I've just used this sample:
>
> https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HConnectionManager.html
>
>  HConnection connection = HConnectionManager.createConnection(config);
>  HTableInterface table = connection.getTable(TableName.valueOf("table1"));
>  try {
>    // Use the table as needed, for a single operation and a single thread
>  } finally {
>    table.close();
>    connection.close();
>  }
>
>
> Functional testing OK, performance fails :((((
>
>
>
> 2014-12-12 19:32 GMT+03:00 Stack <stack@duboce.net>:
> >
> > Does zk count go up on each request to the servlet? Which version of
> hbase
> > so can try on this end?  Do you have some client-side log? Better if you
> > cache the connection rather than make it each time since setup is costly
> > but lets fix first problem first.
> > St.Ack
> >
> > On Fri, Dec 12, 2014 at 2:47 AM, Serega Sheypak <
> serega.sheypak@gmail.com>
> > wrote:
> >
> > > Hi, I'm using HConnectionManager from java servlet
> > > Looks like it's leaking, all my zookeepers complains that there is too
> > many
> > > connections from servlet hosts.
> > > Typical line from lZK log:
> > >
> > > oo many connections from /my.tomcat.server.com  - max is 60
> > >
> > >
> > > Here is code sample
> > >
> > > public class BaseConnection {
> > >
> > >     private static final Logger LOG =
> > > LoggerFactory.getLogger(BaseConnection.class);
> > >
> > >     protected void close(HConnection connection){
> > >         try{
> > >             if(connection == null){
> > >                 return;
> > >             }
> > >             connection.close();
> > >         }
> > >         catch (Exception e){
> > >             LOG.warn("Error while closing HConnection", e);
> > >         }
> > >     }
> > >
> > >     protected void close(HTableInterface hTable){
> > >         try{
> > >             if(hTable == null){
> > >                 return;
> > >             }
> > >             hTable.close();
> > >         }
> > >         catch (Exception e){
> > >             LOG.warn("Error while closing HTable", e);
> > >         }
> > >     }
> > > }
> > >
> > > sample PUT code from subclass:
> > >
> > > public void put(List<MyBean> entries){
> > >     HConnection hConnection = null;
> > >     HTableInterface hTable = null;
> > >     try {
> > >         List<Put> puts = new ArrayList<Put>(entries.size());
> > >         for(MyBean myBean : entries){
> > >             puts.add(new MyBeanSerDe().createPut(myBean));
> > >         }
> > >         hConnection =
> HConnectionManager.createConnection(configuration);
> > >         hTable = hConnection.getTable(NAME_B);
> > >         hTable.put(puts);
> > >
> > >     }catch (Exception e){
> > >         LOG.error("Error while doing bulk put", e);
> > >     }finally{
> > >         close(hTable);
> > >         close(hConnection);
> > >     }
> > > }
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message