hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject something wrong on trunk possibly
Date Sat, 04 Apr 2009 09:19:57 GMT
Hey guys,

There seems to be something wrong on trunk... I used to have long map-reduce
jobs, but now they are failing, unable to commit:

2009-04-04 01:17:09,279 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
locateRegionInMeta attempt 5 of 10 failed; retrying after sleep of 8000
java.io.IOException: HRegionInfo was null or empty in .META.
        at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:566)
        at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:515)
        at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.relocateRegion(HConnectionManager.java:484)
... etc

Basically mappers get stuck up on commits and make no progress, mapred kills
them, done.

I've spent some time banging at it - made sure that ulimit -n is good, set
the ipc handler limit to 30, cranked down the number of maps I'm doing,
etc.  To no avail.

At least I figured out how to debug hadoop jobs a bit.

Anyone have thoughts?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message