hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: WARN add_table: Missing .regioninfo:.. No server address.. what to do?
Date Thu, 26 Aug 2010 06:11:18 GMT
On Wed, Aug 25, 2010 at 11:22 AM, Stuart Smith <stu24mail@yahoo.com> wrote:
> Just curious, though, (if it happens again) - assume the regions were invalid - I don't
know, maybe it was halfway through splitting something and died - but say they're invalid.
>

(See if a failed MR task associated with the bad region.  You could
also tgz' the bad region and we can take a look at it for you.)

> Would the best thing to do in that case be a manual deletion of the hdfs directories
containing the invalid regions? What hbase handle that OK?
>

If its a 'bad' region, should be fine.  There'd be no holes in loaded
table.  But if its not...

> And a side question that ties a lot of my issues together - I finally have a (somewhat)
clean interface that moves the occasional too big file into hdfs, and stores everything else
into hbase - I built this up as a layer in java with a metadata/filestore split in hbase (all
file metadata is in hbase, files are directed to hbase/hdfs based on size).
>
> Is there another project that does this? It seems too handy to be the first time someone
did this... Or does something like this always end up needing domain-specific tweaks &
interfaces?
>

I haven't heard of a project like this (though as you say, you can't
be the first... maybe you are though?)

> Because once you have huge cells in hbase, it really seems to be unhappy. Especially
when a good chunk of your tasks are done as M/R tasks or some layer on top of M/R.
>

Yeah, I'd imagine so.  At least default configuration is set for cells
in the 0-50k or so size.  I'd imagine they'd need to be pulled around
some if cells are MBs.

> Or would this be a good project to open-source? Or pointless to do so?
>

Do it on github as Ted suggests.  It'll either flourish and then
you'll have to figure out how to support it or it'll wither when you
move on (add it to supporting projects on wiki so its easier for folks
to find?)

> I guess in the long-run hbase could absorb these requirements with some tweaks of the
file format, but I thought it could be nice to do this with a little library layer on top.
>

You are a good man Stu,
St.Ack


> --- On Mon, 8/23/10, Stack <stack@duboce.net> wrote:
>
>> From: Stack <stack@duboce.net>
>> Subject: Re: WARN add_table: Missing .regioninfo:.. No server address.. what to do?
>> To: user@hbase.apache.org
>> Date: Monday, August 23, 2010, 6:08 PM
>> On Mon, Aug 23, 2010 at 1:35 PM,
>> Stuart Smith <stu24mail@yahoo.com>
>> wrote:
>> >
>> > Hmm... AFAICT, if the regioninfo files is gone from a
>> region directory (and I looked on hdfs, and it is gone), the
>> region is hosed.
>>
>> Is it a legit region?  Its wholesome looking with
>> hfiles that make
>> sense (non-zero)?  My guess is that the regions are
>> incompletes and
>> loadtable is not smart enough recognizing them as so.
>> If you grep
>> your master log for the region encoded name, do you find
>> anything?
>> Maybe this way you can figure its provenance?
>>
>> St.Ack
>>
>
>
>
>
>

Mime
View raw message