hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Marc Spaggiari <jean-m...@spaggiari.org>
Subject Re: hBase - the server has too many connections (maxClientConn property set to 0 does not help)
Date Tue, 12 Nov 2013 12:13:33 GMT
So this occurs when you run you MR job to index HBase data from Solr? How
many maps  do you have at the same time? Is that more than 30? If so, can
you try to get than lower to see if  that helps? Else you can see if
maxClientCnxns in ZK side can be increased?

JM


2013/11/12 glumet <jan.bouchner@gmail.com>

> Hello,
> I use hbase to store my data which comes from a crawler. These data are
> indexed from hbase to Solr. But I have a problem with hBase. As I have
> larger DB now, everytime I try to index data from hbase the error shown
> below occures. This is the part of log file hadoop.log
>
> There is something like Caused by: This could be a sign that the server has
> too many connections (30 is the default). and I found here on SO that this
> could be beacues of maxClientConnections property so I set up this value to
> 0 but this solution did not help.
>
> /2013-11-11 15:30:22,638 ERROR zookeeper.ClientCnxn - Error while calling
> watcher
> java.lang.NullPointerException
> at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.resetZooKeeperTrackers(HConnectionManager.java:315)
> at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.abort(HConnectionManager.java:1340)
> at
>
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:343)
> at
>
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:261)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:488)
> 2013-11-11 15:30:27,925 WARN  mapred.FileOutputCommitter - Output path is
> null in cleanup
> 2013-11-11 15:30:27,925 WARN  mapred.LocalJobRunner -
> job_local1497313370_0001
> org.apache.gora.util.GoraException: java.lang.RuntimeException:
> org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is able to
> connect to ZooKeeper but the connection closes immediately. This could be a
> sign that the server has too many connections (30 is the default). Consider
> inspecting your ZK server logs for that error and then make sure you are
> reusing HBaseConfiguration as often as you can. See HTable's javadoc for
> more information.
> at
>
> org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:167)
> at
>
> org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:118)
> at
>
> org.apache.gora.mapreduce.GoraOutputFormat.getRecordWriter(GoraOutputFormat.java:88)
> at
>
> org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.<init>(ReduceTask.java:568)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:637)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:398)
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is able to
> connect to ZooKeeper but the connection closes immediately. This could be a
> sign that the server has too many connections (30 is the default). Consider
> inspecting your ZK server logs for that error and then make sure you are
> reusing HBaseConfiguration as often as you can. See HTable's javadoc for
> more information.
> at org.apache.gora.hbase.store.HBaseStore.initialize(HBaseStore.java:127)
> at
>
> org.apache.gora.store.DataStoreFactory.initializeDataStore(DataStoreFactory.java:102)
> at
>
> org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:161)
> ... 6 more
> Caused by: org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is
> able to connect to ZooKeeper but the connection closes immediately. This
> could be a sign that the server has too many connections (30 is the
> default). Consider inspecting your ZK server logs for that error and then
> make sure you are reusing HBaseConfiguration as often as you can. See
> HTable's javadoc for more information.
> at
>
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:155)
> at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:1002)
> at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:304)
> at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:295)
> at
>
> org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:157)
> at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:90)
> at org.apache.gora.hbase.store.HBaseStore.initialize(HBaseStore.java:109)
> ... 8 more
> Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:809)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:837)
> at
>
> org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndFailSilent(ZKUtil.java:903)
> at
>
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:133)
> ... 14 more/
>
> Do you know where is the problem please?
>
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/hBase-the-server-has-too-many-connections-maxClientConn-property-set-to-0-does-not-help-tp4052728.html
> Sent from the HBase User mailing list archive at Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message