hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1342) Add to filesystem info needed to rebuild .META.
Date Thu, 30 Apr 2009 20:53:30 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12704791#action_12704791
] 

stack commented on HBASE-1342:
------------------------------

Currently, I'm thinking of writing a file into the Region subdirectory named regioninfo on
region creation.  It would contain the serialized HRegionInfo followed by a text version of
HRegionInfo.  Cat'ing the regioninfo file, you'd see binary garbage followed by text representation.
 A tool should be able to recreate an HRI just be deserializing from this file.  The text
part would be so humans could see what the region was about.

I was originally thinking of writing this file every time the region was deployed.  This would
up the hdfs load on startup. I was thinking we'd do this because parts of the HRI can go stale
and rewriting it on every deploy would keep it pretty fresh.  But I think as far as rebuilding
the .META., figuring what the table and column descriptors were just before the need for repair
will be figured elsewhere.

A tool that rebuilds the .META. will still not have enough to rebuild the .META. using regioninfo.
 The rebuild tool will have to sort and make sense of the list of regions it has figuring
which are parent regions and which are daughters, etc., but I think that this can be done
up in the tool using common sense checking if storefiles have references, etc.; in other words,
there is enough to go on rebuilding table and column descriptors.  The regioninfo is for region
name, region id and for easy access to start/end key, critical info needed rebuilding .META.

Was also going to change master logging so it emitted the HRI at INFO level so could go there
ultimately to resolve what happened when -- if needed.

> Add to filesystem info needed to rebuild .META.
> -----------------------------------------------
>
>                 Key: HBASE-1342
>                 URL: https://issues.apache.org/jira/browse/HBASE-1342
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: stack
>             Fix For: 0.20.0
>
>
> From Cosmin up on list:
> {code}
> Hi,
> Is there any way to rebuild the .META. Table with information in HDFS?
> Long story short, the namenode ran out of disk and crashed and we couldn't
> have it load the edits.new file at startup (the edits.new file seemed
> corrupted and it failed to read some Long value) so we removed the edits.new
> file. Apparently the edits file had 10 - 15 entries. Some edit data on
> -ROOT-, some edit data on .META., some data on a table compaction
> (comapction.dir) and some data on some hbase logs. Only last entry seem to
> be bogus, having almost a megabyte of trailing zeroes.
> After renaming edits.new HDFS started ok and HBase data is present. However
> when we started Hbase one table was missing entirely and one table was
> empty. The HBase master report shows Regions On FS: 94, but the report on
> region servers shows only 10 regions on all the region servers.
> Is there any way to cleanup hbase and rebuild the .META. table from HDFS?
> This would be ok even if it implies throwing up some data.
> If we would clean the edits.new file and restart HDFS or if we would
> manually apply the edits would that still make consistent changes to HBase
> data?
> Thanks,
> Cosmin
> {code}
> Lets log the info needed to rebuild your .META. or add things like HRegionInfo made creating
a region as a file under each region directory.  That don't seem hard and the info could be
used rebuilding.  What else would we need?
> Marking as blocker/critical for 0.20.0.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message