hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: distcp hbase-0.94.6(CDH4.5) hfiles to hbase1.2(CDH5.8)?
Date Mon, 15 Aug 2016 18:01:14 GMT
Please verify that your 0.94 cluster is configured with hfile v2.
Config hfile.format.version should have value of 2.

To obtain region boundaries, you can use 'describe' hbase shell command.

Cheers

On Mon, Aug 15, 2016 at 9:48 AM, Rob Verkuylen <rob@verkuylen.net> wrote:

> The 2nd cluster is an operational cluster, so option 2 seems to fit nicely.
>
> Will the LoadIncrementalHFiles on the Hbase1.2 cluster be able to process
> the 0.94 version Hfiles?
>
> Also is there a convenient way to get the region splits from the old
> hfiles, or is going though them the only option?
>
> On Sun, Aug 14, 2016 at 9:00 PM, Jerry He <jerryjch@gmail.com> wrote:
>
> > If you distcp the raw hfiles, you have a couple of options to restore the
> > data on the second cluster.
> >
> > 1. You an copy the entire hbase root.dir, you can set the hbase root.dir
> to
> > this directory and bootstrap the new cluster from there.
> >     Before you start the new cluster, run the 'hbase upgrade' on the
> hbase
> > root.dir.
> >     Refer to "13.1.3. Upgrading to 1.0 from 0.94"
> > https://hbase.apache.org/book.html#_upgrade_paths
> >
> > 2.  You can run the bulk load tool to load the copied hfiles into the new
> > cluster.
> >      Refer to "130.12. CompleteBulkLoad" in the HBase Reference book.
> >      You can write a program or script to go through all the region
> > directories.
> >
> > Option 1 is more delicate, but as you said the old hdfs was fine, it
> should
> > work for you.
> > For option 2, pre-split the tables on the new cluster to match the region
> > boundaries of the old tables.
> >
> > Jerry
> >
> > On Sun, Aug 14, 2016 at 8:08 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> > > For the Import tool, you can specify the following (quoted from usage):
> > >
> > >     System.err.println("To import data exported from HBase 0.94, use");
> > >
> > >     System.err.println("  -Dhbase.import.version=0.94");
> > >
> > > FYI
> > >
> > > On Sun, Aug 14, 2016 at 12:09 AM, Rob Verkuylen <rob@verkuylen.net>
> > wrote:
> > >
> > > > We're recovering from a crash of hbase-0.94.6 where the master
> refuses
> > to
> > > > come up, while hdfs is fine. Why this happened exactly is something
> > we're
> > > > looking into.
> > > >
> > > > In the meantime we're looking to migrate as fast as possible to
> another
> > > > cluster running hbase1.2.
> > > >
> > > > Since we cannot use the default import tool anymore. If we distcp the
> > raw
> > > > hfiles from old to new cluster, is there a way to safely
> load/bulkload
> > > > these into the hbase1.2 cluster?
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message