hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: java.io.IOException: HRegionInfo was null or empty in .META.
Date Tue, 14 Jul 2009 22:14:33 GMT
Try doing this on the shell:

flush '.META.'
major_compact '.META.'

did you use 'truncate' command at any point?

On Tue, Jul 14, 2009 at 3:10 PM, Haijun Cao<haijuncao@ymail.com> wrote:
>
> Hi
>
> I am running hbase PE test, loaded TestTable with 10M records without any problem, later
hbase crashed (during another sequentialWrite test), after restart, I can't count TestTable,
it always get stuck at 1.1 mil records.
>
>
> I am wondering if anybody has encountered the same problem (data got corrupted after
server crash)? How do you recover? At this point, it is just test data, so it is ok for me
to lose some data. Is there a way to drop/repair the bad region (0001192603)? Does hbase have
a auto-repair tool like fsck?
>
> Your help is greatly appreciated.
>
> Haijun
>
>
> Error stack:
>
> Current count: 1100000, row: 0001099999
> NativeException: java.lang.RuntimeException: org.apache.hadoop.hbase.client.RetriesExhaustedException:
Trying to contact region server null for region , row '0001192603', but failed after 5 attempts.
> Exceptions:
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
>
>        from org/apache/hadoop/hbase/client/HTable.java:2083:in `hasNext'
>        from sun.reflect.GeneratedMethodAccessor9:-1:in `invoke'
>        from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
>        from java/lang/reflect/Method.java:597:in `invoke'
>        from org/jruby/javasupport/JavaMethod.java:298:in `invokeWithExceptionHandling'
>        from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
>        from org/jruby/java/invokers/InstanceMethodInvoker.java:36:in `call'
>        from org/jruby/runtime/callsite/CachingCallSite.java:70:in `call'
>        from org/jruby/ast/CallNoArgNode.java:61:in `interpret'
>        from org/jruby/ast/WhileNode.java:127:in `interpret'
>        from org/jruby/ast/NewlineNode.java:104:in `interpret'
>        from org/jruby/ast/BlockNode.java:71:in `interpret'
>        from org/jruby/internal/runtime/methods/InterpretedMethod.java:163:in `call'
>        from org/jruby/internal/runtime/methods/DefaultMethod.java:144:in `call'
>        from org/jruby/runtime/callsite/CachingCallSite.java:273:in `cacheAndCall'
>        from org/jruby/runtime/callsite/CachingCallSite.java:112:in `call'
>
>
> I scan the .META. table and noticed that region 0001192603 has no info columns, only
historian columns. I also check the hadoop file system, region 0001192603 has a oldlogfile.log
file in its directory (other regions don't):
>
> Found 3 items
> -rw-r--r--   3 bamboo supergroup        619 2009-07-13 16:28 /user/bamboo/hbase/TestTable/954997373/.regioninfo
> drwxr-xr-x   - bamboo supergroup          0 2009-07-13 16:28 /user/bamboo/hbase/TestTable/954997373/info
> -rw-r--r--   3 bamboo supergroup   39417817 2009-07-13 16:52 /user/bamboo/hbase/TestTable/954997373/oldlogfile.log
>
>
>

Mime
View raw message