hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gautam <gautamkows...@gmail.com>
Subject Re: Copying data from 94 to 98 ..
Date Tue, 16 Sep 2014 05:36:21 GMT
14/09/15 23:34:59 DEBUG snapshot.SnapshotManifestV1: Adding reference for
file (4/4): hftp://
master42.stg.com:50070/hbase/.hbase-snapshot/msg_snapshot/84f60fc2aa7e96df91e6289e6c19dc25/c/afe341e4149649578c5861e32494dbec

14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export failed
java.io.IOException
at
org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145)
at
org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265)
at
org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119)
at
org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125)
at
org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitReferencedFiles(SnapshotReferenceUtil.java:108)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot.getSnapshotFiles(ExportSnapshot.java:479)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot.access$200(ExportSnapshot.java:89)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportSnapshotInputFormat.getSplits(ExportSnapshot.java:600)
at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1107)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1124)
at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:178)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1023)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:976)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:976)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:582)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:612)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot.runCopyJob(ExportSnapshot.java:751)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:905)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:975)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:979)
Caused by: java.io.IOException: read=-1, wanted=4
at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175)
at
org.apache.hadoop.hbase.regionserver.StoreFileInfo.<init>(StoreFileInfo.java:115)
at
org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java:204)
at
org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.buildManifestFromDisk(SnapshotManifestV1.java:179)
at
org.apache.hadoop.hbase.snapshot.SnapshotManifestV1$1.call(SnapshotManifestV1.java:131)
at
org.apache.hadoop.hbase.snapshot.SnapshotManifestV1$1.call(SnapshotManifestV1.java:127)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)

On Mon, Sep 15, 2014 at 10:18 PM, Matteo Bertozzi <theo.bertozzi@gmail.com>
wrote:

> can you post the full exception and the file path ?
> maybe there is a bug in looking up the reference file.
> It seems to not be able to find enough data in the file...
>
> Matteo
>
>
> On Mon, Sep 15, 2014 at 10:08 PM, Gautam <gautamkowshik@gmail.com> wrote:
>
> > Thanks for the reply Matteo.
> >
> > This is exactly what I did. I modified the source cluster's dir structure
> > to mimic that of the 98 cluster. I even got as far as it trying to look
> > through the reference files.
> >
> > I end up with this exception :
> >
> > 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export failed
> > java.io.IOException
> > at
> >
> >
> org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145)
> > at
> >
> >
> org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265)
> > at
> >
> >
> org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119)
> > at
> >
> >
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125)
> > ...
> > ..
> > Caused by: java.io.IOException: read=-1, wanted=4
> > at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175)
> > at
> >
> >
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.<init>(StoreFileInfo.java:115)
> > at
> >
> >
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java
> > ..
> >
> >
> > This and Ted's reply about HBASE-7987 leads me to believe that the export
> > tool from my distro is in capable of working around the regionManifest
> file
> > requirement. I'm now left with the option of downgrading my dest cluster
> to
> > 94, copying data and then upgrading using the upgrade migration tool.
> > Wanted to know if others have tried this or there are other things I can
> > do. If not, i'l just go ahead and do this :-)
> >
> >
> > Cheers,
> > -Gautam.
> >
> >
> >
> > On Mon, Sep 15, 2014 at 8:10 PM, Matteo Bertozzi <
> theo.bertozzi@gmail.com>
> > wrote:
> >
> > > 94 and 98 differs in directory layout
> > > so 98 is not able to read 94 layout unless you run the migration tool
> > > which is basically moving all the data in a "default" namespace
> directory
> > > e.g.
> > > /hbase/table -> /hbase/data/default/table
> > > /hbase/.archive/table -> /hbase/archive/default/table
> > >
> > > Matteo
> > >
> > >
> > > On Mon, Sep 15, 2014 at 6:17 PM, Gautam <gautamkowshik@gmail.com>
> wrote:
> > >
> > > > Yep, looks like the CDH distro backports HBASE-7987. Having said
> that,
> > is
> > > > there a transition path for us or are we hosed :-) ? In general,
> what's
> > > the
> > > > recommended way to achieve this, at this point I feel i'm going
> around
> > > the
> > > > system to achieve what I want. If nothing else works with export
> > snapshot
> > > > should I just downgrade to 94, export snapshot and then upgrade to
> 98?
> > Is
> > > > the upgrade migration path different from what export snapshot does
> > (i'd
> > > > imagine yes)?
> > > >
> > > > Cheers,
> > > > -Gautam.
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, Sep 15, 2014 at 5:14 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> > > >
> > > > > bq. 98.1 on dest cluster
> > > > >
> > > > > Looking at the history for SnapshotManifestV1, it came with
> > HBASE-7987
> > > > > which went to 0.99.0
> > > > >
> > > > > Perhaps you're using a distro with HBASE-7987 ?
> > > > >
> > > > > On Mon, Sep 15, 2014 at 4:58 PM, Gautam <gautamkowshik@gmail.com>
> > > wrote:
> > > > >
> > > > > > Hello,
> > > > > >           I'm trying to copy data between Hbase clusters on
> > different
> > > > > > versions. I am using :
> > > > > >
> > > > > > /usr/bin/hbase  org.apache.hadoop.hbase.snapshot.ExportSnapshot
> > > > > >   -chuser hbase
> > > > > >   -chgroup hadoop
> > > > > >   -snapshot msg_snapshot
> > > > > >   -mappers 50
> > > > > >   -copy-from hftp://src-cluster:50070/hbase
> > > > > >   -copy-to hdfs:/dest-cluster:8020/hbase
> > > > > >
> > > > > >
> > > > > > Till now, based on various tips from the mailing list, I have
> > > modified
> > > > > the
> > > > > > source cluster data dir paths to mimic the 98 convention
> (archive,
> > > > table
> > > > > > data paths, etc). This helped in jumping some roadblocks but
not
> > all.
> > > > > >
> > > > > > This is what I see now :
> > > > > >
> > > > > >
> > > > > > 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export
> > > failed
> > > > > > java.io.IOException
> > > > > > at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145)
> > > > > > at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265)
> > > > > > at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119)
> > > > > > at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125)
> > > > > > ...
> > > > > > ..
> > > > > > Caused by: java.io.IOException: read=-1, wanted=4
> > > > > > at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175)
> > > > > > at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.<init>(StoreFileInfo.java:115)
> > > > > > at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java
> > > > > > ..
> > > > > >
> > > > > >
> > > > > > Fails while trying to read refernce hfile. Is this something
> folks
> > > have
> > > > > > done before and/or is possible to do? I'd really like to do
this
> > > > without
> > > > > > having to upgrade my source cluster or downgrade my dest cluster.
> > > > > >
> > > > > > I'm using 94.6 on source cluster and 98.1 on dest cluster.
> > > > > >
> > > > > >
> > > > > > Cheers,
> > > > > > -Gautam.
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > "If you really want something in this life, you have to work for it.
> > Now,
> > > > quiet! They're about to announce the lottery numbers..."
> > > >
> > >
> >
> >
> >
> > --
> > "If you really want something in this life, you have to work for it. Now,
> > quiet! They're about to announce the lottery numbers..."
> >
>



-- 
"If you really want something in this life, you have to work for it. Now,
quiet! They're about to announce the lottery numbers..."

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message