hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jon Graham <sjclou...@gmail.com>
Subject Re: Table recovery options
Date Thu, 24 Sep 2009 21:27:21 GMT
Hello,

I am also interested in data recovery methods/tools.

I do not understand the relationship between the HBase files stored in
Hadoop and the
logical organization of the HBase information.

Is there any documentation that describes the HBase disk file layout and
format of the
-ROOT-, META, region info and user TABLE structures?

This would very helpful in trying to create a recovery or data base checking
tool.

Thanks for help,
Jon


On Thu, Sep 24, 2009 at 1:18 PM, stack <stack@duboce.net> wrote:

> On Wed, Sep 23, 2009 at 4:15 PM, elsif <elsif.then@gmail.com> wrote:
>
> >
> > We have a couple clusters running with lzo compression.  When testing
> > the new 0.20.1 release
>
>
> You mean hadoop's new 0.20.1 release?
>
>
>
> > I setup a single node cluster and reused the
> > compression jar and native libraries from the 0.20.0 release.
>
>
>
> hadoop 0.20.0 release?
>
> What release of hbase are you using?
>
>
>
> > The
> > following session log shows a table being created with the lzo option
> > and some rows being added.  After hbase is restarted the table is no
> > longer accessible - the region server crashed during the flush operation
> > due to a SIGFPE.
> >
> > The flush that was done during shutdown?  The flush of the .META.?  If
> this
> failed, then state of .META. would not have been persisted and yes, you
> would have lost your table.
>
>
>
> > Would it be possible to add a check to verify the compression feature
> > before it is used in a table to avoid corruption?  A simple shell or cli
> > option would be great.
> >
>
> Sounds like a good idea.  What would you suggest?  You could force a flush
> on a table with data on it and check if it worked or not?
>
>
>
> >
> > In general, once hbase tables are corrupted is there anyway to repair
> > them?  - In this test case the table is never written to disk.
> >
> >
> Depends on the 'corruption'.  Generally yes, there are ways.  Below it
> seems
> a compression library mismatch is preventing hbase writing the filesystem.
> Can you fix this and retry?
>
>
>
> > Is it possible to regenerate an hbase table from the data files stored
> > in hdfs?
> >
>
>
> Yes. You'd have to write a script.   In hdfs, under each region directory
> there is a file named .regioninfo.  It has the content of the .META. table
> for this region serialized.  A script could look at some subset of the
> regions on disk -- say all that make up a table and do fix up of .META.  On
> the next scan of .META. the table should be onlined.  Let me know if you'd
> like some help with this and we can work on it together.
>
>
>
> >
> > Are there any preventative measures we can take to make it easier to
> > roll back to a valid state?
> >
>
> You can backup hbase content if its small, or you can scan content from
> before the date at which invalid data shows.  What else would you like?
>
>  St.Ack
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message