hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: hbase crashed, table missing
Date Thu, 03 Dec 2009 20:21:27 GMT
What's in the HDFS folder of that table? Here I see that you should
have something like:

/hbase/cached_web_pages/1325672518/http/  stuff...

Was there only this one region?

Also are you able to enable a region in the shell? Take one of the row
key from .META. and do

> enable_region 'region name'

J-D

On Thu, Dec 3, 2009 at 12:11 PM, mike anderson <saidtherobot@gmail.com> wrote:
> Here's a snippit from the meta table (I can send you the whole thing, but
> it's quite large),
>
> cached_web_pages,http%3A%2F column=info:serverstartcode,
> timestamp=1259853027975, value=1259852967063
>  %2Fdx.doi.org%2F10.1002%252
>
>  Fajpa.21214,1259739437144
>
>  cached_web_pages,http%3A%2F column=historian:assignment,
> timestamp=1259807436758, value=Region assigned to se
>  %2Fdx.doi.org%2F10.1002%252 rver
> ghetto169.projectlounge.com,60020,1256139356112
>
>  Fejoc.200900768,12555040994
>
>  35
>
>  cached_web_pages,http%3A%2F column=historian:open, timestamp=1259807436723,
> value=Region opened on server : g
>  %2Fdx.doi.org%2F10.1002%252 hetto169.projectlounge.com
>
>  Fejoc.200900768,12555040994
>
>  35
>
>  cached_web_pages,http%3A%2F column=historian:assignment,
> timestamp=1259853024917, value=Region assigned to se
>  %2Fdx.doi.org%2F10.1002%252 rver
> ghetto167.projectlounge.com,60020,1259852967063
>
>  Fsmi.1285,1258589376676
>
>  cached_web_pages,http%3A%2F column=historian:open, timestamp=1259853027984,
> value=Region opened on server : g
>  %2Fdx.doi.org%2F10.1002%252 hetto167.projectlounge.com
>
>  Fsmi.1285,1258589376676
>
>  cached_web_pages,http%3A%2F column=info:regioninfo,
> timestamp=1258589203875, value=REGION => {NAME => 'cached
>  %2Fdx.doi.org%2F10.1002%252 _web_pages,http\\x253A\\x252F\\x252Fdx.doi.org
> \\x252F10.1002\\x25252Fsmi.1285,125
>  Fsmi.1285,1258589376676     8589376676', STARTKEY => 'http\\x253A\\x252F\\
> x252Fdx.doi.org\\x252F10.1002\\x252
>                             52Fsmi.1285', ENDKEY => 'http\\x253A\\x252F\\
> x252Fdx.doi.org\\x252F10.1016\\x252F
>                             j.apergo.2009.09.005', ENCODED => 1325672518,
> TABLE => {{NAME => 'cached_web_page
>                             s', FAMILIES => [{NAME => 'http', VERSIONS
=>
> '1', COMPRESSION => 'NONE', TTL =>
>                             '2147483647', BLOCKSIZE => '65536', IN_MEMORY
> => 'false', BLOCKCACHE => 'true'}]}
>                             }
>
>
> and you can see the table which has gone missing 'cached_web_pages' in the
> key spot. The crash over the weekend was pretty traumatic. Complete power
> outage to the entire cluster except(!) for the master.  The data is
> definitely still on HDFS, I will take a look at the add_table script and
> upgrade to 0.20.2.
>
>
> Cheers and thanks a lot.
>
> mike
>
>
> On Thu, Dec 3, 2009 at 2:51 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:
>
>> This is weird if the table is in .META. and still not showing up...
>> could you pastebin the .META. rows?
>>
>> Also was it a new table that was just created or has it been there for
>> some time?
>>
>> What kind of crash did you get this weekend?
>>
>> The best way to recover your data, if it's still on HDFS, will be to
>> upgrade to 0.20.2 and use the script bin/add_table.rb to rebuild
>> .META.
>>
>> J-D
>>
>> On Thu, Dec 3, 2009 at 11:29 AM, mike anderson <saidtherobot@gmail.com>
>> wrote:
>> > From the web UI and from calling 'list' in the shell I can't see the
>> table
>> > name.
>> >
>> > Hadoop/Hbase 0.20/0.20.1, distributed setup, 10 nodes.
>> >
>> > -mike
>> >
>> > On Thu, Dec 3, 2009 at 1:54 PM, Jean-Daniel Cryans <jdcryans@apache.org
>> >wrote:
>> >
>> >> Mike,
>> >>
>> >> So if you looked in .META. and the rows are there, how did you figure
>> >> that the table is missing?
>> >>
>> >> Also the usuals: which version of Hadoop/HBase, what kind of setup, etc
>> >>
>> >> J-D
>> >>
>> >> On Thu, Dec 3, 2009 at 7:29 AM, mike anderson <saidtherobot@gmail.com>
>> >> wrote:
>> >> > Hbase crashed on me this weekend, and upon restarting one of the
>> tables
>> >> is
>> >> > just completely gone. All of the table data is still in HDFS and my
>> >> missing
>> >> > table is still mentioned in .META.. I tried restarting hbase a few
>> times,
>> >> > but the table didn't show up. What else can I do to debug this? I
>> looked
>> >> > through the logs, but nothing really jumped out at me. Is there
>> something
>> >> I
>> >> > should look for?
>> >> >
>> >> > I took a look at this ticket,
>> >> > http://issues.apache.org/jira/browse/HBASE-1342, but don't know
>> enough
>> >> about
>> >> > the inner workings of hbase to make sense of it.
>> >> >
>> >> >
>> >> > thanks in advance.
>> >> >
>> >>
>> >
>>
>

Mime
View raw message