hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Mackles <pmack...@adobe.com>
Subject Re: backup strategies
Date Thu, 16 Aug 2012 11:53:09 GMT
Hi Rita

By default, the export that ships with hbase writes KeyValue objects to a
sequence file. It is a very simple app and it wouldn't be hard to roll
your own export program to write to whatever format you wanted (its a very
simple app). You can use the current export program as a basis and just
change the output of the mapper.

I will say that I spent a lot of time thinking about backups and DR and I
didn't really worry much about hbase versions. The file formats for hbase
don't change that often and when they do, there is usually a pretty
straight-forward conversion process. Also, if you are doing something like
full daily backups then I am having trouble imagining a scenario where you
would need to restore from anything but the most recent backup.

Depending on which version of hbase you are using, there are probably much
bigger issues with using export for backups that you should worry about
like being able to restore in a timely fashion, preserving deletes and
impact of the backup procress on your SLA.


On 8/16/12 7:31 AM, "Rita" <rmorgan466@gmail.com> wrote:

>I am sure this topic has been visited many times but I though I ask to see
>if anything changed.
>We are using hbase with close to 40b rows and backing up the data is
>non-trivial. We can use export table to another Hadoop/HDFS filesystem but
>I am not aware of any guaranteed way of preserving data from one version
>Hbase to another (specifically if its very old) . Is there a program which
>will serialize the data into JSON/XML and dump it on a Unix filesystem?
>Once I get the data we can compress it whatever we like and back it up
>using our internal software.
>--- Get your facts first, then you can distort them as you please.--

View raw message