hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matteo Bertozzi <theo.berto...@gmail.com>
Subject Re: Copying data from 94 to 98 ..
Date Tue, 16 Sep 2014 03:10:33 GMT
94 and 98 differs in directory layout
so 98 is not able to read 94 layout unless you run the migration tool
which is basically moving all the data in a "default" namespace directory
e.g.
/hbase/table -> /hbase/data/default/table
/hbase/.archive/table -> /hbase/archive/default/table

Matteo


On Mon, Sep 15, 2014 at 6:17 PM, Gautam <gautamkowshik@gmail.com> wrote:

> Yep, looks like the CDH distro backports HBASE-7987. Having said that, is
> there a transition path for us or are we hosed :-) ? In general, what's the
> recommended way to achieve this, at this point I feel i'm going around the
> system to achieve what I want. If nothing else works with export snapshot
> should I just downgrade to 94, export snapshot and then upgrade to 98? Is
> the upgrade migration path different from what export snapshot does (i'd
> imagine yes)?
>
> Cheers,
> -Gautam.
>
>
>
>
> On Mon, Sep 15, 2014 at 5:14 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > bq. 98.1 on dest cluster
> >
> > Looking at the history for SnapshotManifestV1, it came with HBASE-7987
> > which went to 0.99.0
> >
> > Perhaps you're using a distro with HBASE-7987 ?
> >
> > On Mon, Sep 15, 2014 at 4:58 PM, Gautam <gautamkowshik@gmail.com> wrote:
> >
> > > Hello,
> > >           I'm trying to copy data between Hbase clusters on different
> > > versions. I am using :
> > >
> > > /usr/bin/hbase  org.apache.hadoop.hbase.snapshot.ExportSnapshot
> > >   -chuser hbase
> > >   -chgroup hadoop
> > >   -snapshot msg_snapshot
> > >   -mappers 50
> > >   -copy-from hftp://src-cluster:50070/hbase
> > >   -copy-to hdfs:/dest-cluster:8020/hbase
> > >
> > >
> > > Till now, based on various tips from the mailing list, I have modified
> > the
> > > source cluster data dir paths to mimic the 98 convention (archive,
> table
> > > data paths, etc). This helped in jumping some roadblocks but not all.
> > >
> > > This is what I see now :
> > >
> > >
> > > 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export failed
> > > java.io.IOException
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125)
> > > ...
> > > ..
> > > Caused by: java.io.IOException: read=-1, wanted=4
> > > at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.<init>(StoreFileInfo.java:115)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java
> > > ..
> > >
> > >
> > > Fails while trying to read refernce hfile. Is this something folks have
> > > done before and/or is possible to do? I'd really like to do this
> without
> > > having to upgrade my source cluster or downgrade my dest cluster.
> > >
> > > I'm using 94.6 on source cluster and 98.1 on dest cluster.
> > >
> > >
> > > Cheers,
> > > -Gautam.
> > >
> >
>
>
>
> --
> "If you really want something in this life, you have to work for it. Now,
> quiet! They're about to announce the lottery numbers..."
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message