hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco Gallotta <ma...@gallotta.co.za>
Subject Re: Table listed in "list", but not in .META.
Date Fri, 10 Aug 2012 22:08:01 GMT
Here's the output from hbck -details: http://pastebin.com/ZxVZEctY 

Extract:

6 inconsistencies detected. 
Status: INCONSISTENT

6 is the number of tables that appear in "list" but cannot be operated on (which btw, includes
not being able to run disable/drop on them - both ops say table not found). I also just noticed
"foo" does not occur in a table list, although I did create it at one point but was able to
clear it from .META. when it also was reporting table not found when trying to disable/drop
it. All these come from when I ^C'ed (i.e. killed) table creation when I was trying to get
lzo compression working and table creation was hanging.

Is there any way to repair this? I see hbck has repair options, but I want to proceed with
caution. 

-- 
Marco Gallotta | Mountain View, California
Software Engineer, Infrastructure | Loki Studios
fb.me/marco.gallotta | twitter.com/marcog
marco@gallotta.co.za | +1 (650) 417-3313

Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Friday 10 August 2012 at 2:49 PM, anil gupta wrote:

> Hi Marco,
> 
> Did anything disastrous happen to cluster?
> Can you try using hbck utility of HBase.
> Run: 'hbase hbck -help' to get all the available options.
> 
> ~Anil
> 
> On Fri, Aug 10, 2012 at 2:22 PM, Marco Gallotta <marco@gallotta.co.za (mailto:marco@gallotta.co.za)>wrote:
> 
> > Hi there
> > 
> > I have a few tables which show up in a "list" in the shell, but produce
> > "table not found" when performing any operation on them. There is no
> > reference of them in the .META. table. It seems to be resulting in some of
> > the hbase services being killed every so often.
> > 
> > Here are some logs from master (foo is one of the tables not found):
> > 
> > 2012-08-09 20:40:44,301 FATAL org.apache.hadoop.hbase.master.HMaster:
> > Master server abort: loaded coprocessors are: []
> > 2012-08-09 20:40:44,301 FATAL org.apache.hadoop.hbase.master.HMaster:
> > Unexpected state : foo,,1343175078663.527bb34f4bb5e40dd42e82054d7c5485.
> > state=PENDING_OPEN, ts=1344570044277,
> > server=ip-10-170-150-10.us-west-1.compute.internal,60020,1344559455110 ..
> > Cannot transit it to OFFLINE.
> > 
> > 
> > There are also a number of the following types of error logs:
> > 
> > 2012-08-09 20:10:04,308 ERROR
> > org.apache.hadoop.hbase.master.AssignmentManager: Failed assignment in:
> > ip-10-170-150-10.us-west-1.compute.internal,60020,1344559455110 due to
> > org.apache.hadoop.hbase.regionserver.RegionAlreadyInTransitionException:
> > Received:OPEN for the
> > region:foo,,1343175078663.527bb34f4bb5e40dd42e82054d7c5485. ,which we are
> > already trying to OPEN.
> > 
> > Any ideas how to find and remove any references to these non-existent
> > tables?
> > 
> > --
> > Marco Gallotta | Mountain View, California
> > Software Engineer, Infrastructure | Loki Studios
> > fb.me/marco.gallotta (http://fb.me/marco.gallotta) | twitter.com/marcog (http://twitter.com/marcog)
> > marco@gallotta.co.za (mailto:marco@gallotta.co.za) | +1 (650) 417-3313
> > 
> > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> 
> 
> -- 
> Thanks & Regards,
> Anil Gupta
> 
> 



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message