hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Bender <jonathan.ben...@gmail.com>
Subject Zookeeper connection error on mapreduce HBase writes
Date Fri, 25 Mar 2011 18:16:02 GMT
Hello all,

I wrote a routine that scans an HBase table, and writes to another table
from within the map function using HTable.put().  When I run the job, it
works fine for the first few rows but ZooKeeper starts having issues opening
up a connection after a while.

Am I just overloading the ZK server by opening up a new HTable connection
during each map() task, or is there something else wrong with my
configuration?  Any other suggestions for reading and writing tables
directly via MapReduce?

Here's the syslog from my tasktracker node: http://pastebin.com/VzUs3TJP

And the error log: http://pastebin.com/9gBRyR4e

And here's the MapReduce code I am running: http://pastebin.com/ir3yWaR1

Excerpt of the error:

org.apache.hadoop.hbase.ZooKeeperConnectionException:
org.apache.hadoop.hbase.ZooKeeperConnectionException:
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /hbase
 at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:988)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:301)
 at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:292)
at
org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:155)
 at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:167)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:145)
 at
com.mydomain.emir.mapreduce.DomainTableTransform$myMapper.map(DomainTableTransform.java:82)
at
com.mydomain.emir.mapreduce.DomainTableTransform$myMapper.map(DomainTableTransform.java:1)
 at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:646)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
at org.apache.hadoop.mapred.Child$4.run(Child.java:240)
 at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at org.apache.hadoop.mapred.Child.main(Child.java:234)
Caused by: org.apache.hadoop.hbase.ZooKeeperConnectionException:
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /hbase
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:147)
 at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:986)
... 15 more

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message