hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco Gallotta <ma...@gallotta.co.za>
Subject Re: Table listed in "list", but not in .META.
Date Sat, 11 Aug 2012 03:21:15 GMT
w00t! I looked in "/hbase/" on the hdfs and found directories referencing the stale tables
and deleting them worked. It now says 0 inconsistencies. Maybe worth adding this to the hbck
-repair script? Anyway, thanks everyone for the help! Hopefully this stops the sporadic crashes
I was seeing. 

-- 
Marco Gallotta | Mountain View, California
Software Engineer, Infrastructure | Loki Studios
fb.me/marco.gallotta | twitter.com/marcog
marco@gallotta.co.za | +1 (650) 417-3313

Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Friday 10 August 2012 at 5:20 PM, Marco Gallotta wrote:

> I never did that, but I mentioned above that all these come from when I ^C'ed (i.e. killed)
table creation when I was trying to get lzo compression working and table creation was hanging.
Where would I check for references to these tables in ZK?
> 
> -- 
> Marco Gallotta | Mountain View, California
> Software Engineer, Infrastructure | Loki Studios
> fb.me/marco.gallotta (http://fb.me/marco.gallotta) | twitter.com/marcog (http://twitter.com/marcog)
> marco@gallotta.co.za (mailto:marco@gallotta.co.za) | +1 (650) 417-3313
> 
> Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> 
> 
> On Friday 10 August 2012 at 5:15 PM, lars hofhansl wrote:
> 
> > Did you blow away your cluster once (in the sense of the deleting all data and starting
from scratch)?This could be left-over data in ZK.
> > 
> > 
> > 
> > ----- Original Message -----
> > From: Marco Gallotta <marco@gallotta.co.za (mailto:marco@gallotta.co.za)>
> > To: user@hbase.apache.org (mailto:user@hbase.apache.org)
> > Cc: 
> > Sent: Friday, August 10, 2012 2:22 PM
> > Subject: Table listed in "list", but not in .META.
> > 
> > Hi there
> > 
> > I have a few tables which show up in a "list" in the shell, but produce "table not
found" when performing any operation on them. There is no reference of them in the .META.
table. It seems to be resulting in some of the hbase services being killed every so often.
> > 
> > Here are some logs from master (foo is one of the tables not found):
> > 
> > 2012-08-09 20:40:44,301 FATAL org.apache.hadoop.hbase.master.HMaster: Master server
abort: loaded coprocessors are: []
> > 2012-08-09 20:40:44,301 FATAL org.apache.hadoop.hbase.master.HMaster: Unexpected
state : foo,,1343175078663.527bb34f4bb5e40dd42e82054d7c5485. state=PENDING_OPEN, ts=1344570044277,
server=ip-10-170-150-10.us-west-1.compute.internal,60020,1344559455110 .. Cannot transit it
to OFFLINE.
> > 
> > 
> > There are also a number of the following types of error logs:
> > 
> > 2012-08-09 20:10:04,308 ERROR org.apache.hadoop.hbase.master.AssignmentManager:
Failed assignment in: ip-10-170-150-10.us-west-1.compute.internal,60020,1344559455110 due
to org.apache.hadoop.hbase.regionserver.RegionAlreadyInTransitionException: Received:OPEN
for the region:foo,,1343175078663.527bb34f4bb5e40dd42e82054d7c5485. ,which we are already
trying to OPEN.
> > 
> > Any ideas how to find and remove any references to these non-existent tables? 
> > 
> > -- 
> > Marco Gallotta | Mountain View, California
> > Software Engineer, Infrastructure | Loki Studios
> > fb.me/marco.gallotta (http://fb.me/marco.gallotta) | twitter.com/marcog (http://twitter.com/marcog)
> > marco@gallotta.co.za (mailto:marco@gallotta.co.za) | +1 (650) 417-3313
> > 
> > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) 
> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message