hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matteo Bertozzi <theo.berto...@gmail.com>
Subject Re: Import HBase snapshots possible?
Date Thu, 01 Aug 2013 14:14:56 GMT
you have to use 3 slashes otherwise is interpreted as local file-system path
-Dhbase.rootdir=hdfs:///10.209.17.88:9000/hbase

Matteo



On Thu, Aug 1, 2013 at 3:09 PM, Siddharth Karandikar <
siddharth.karandikar@gmail.com> wrote:

> Tried what you suggested. Here is what I get -
>
> ssk01:~/siddharth/tools/hbase-0.95.1-hadoop1 # ./bin/hbase
> org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dhbase.rootdir=hdfs://10.209.17.88:9000/hbase -snapshot s1 -copy-to
> /root/siddharth/tools/hbase-0.95.1-hadoop1/data/
> Exception in thread "main" java.lang.IllegalArgumentException: Wrong
> FS: hdfs://10.209.17.88:9000/hbase/.hbase-snapshot/s1/.snapshotinfo,
> expected: file:///
>         at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
>         at
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
>         at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
>         at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
>         at
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
>         at
> org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
>         at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
>         at
> org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils.readSnapshotInfo(SnapshotDescriptionUtils.java:296)
>         at
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.getSnapshotFiles(ExportSnapshot.java:371)
>         at
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:618)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:690)
>         at
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:694)
>
>
> Am I missing something?
>
>
> Thanks,
> Siddharth
>
>
> On Thu, Aug 1, 2013 at 7:31 PM, Matteo Bertozzi <theo.bertozzi@gmail.com>
> wrote:
> > Ok, so to export a snapshot from your HBase cluster, you can do
> > $ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
> > -snapshot MySnapshot -copy-to hdfs:///srv2:8082/my-backup-dir
> >
> > Now on cluster2, hdfs:///srv2:8082 you've your my-backup-dir that
> contains
> > the exported snapshot (note that the snapshot is under hidden dirs
> > .snapshots, and .archive)
> >
> > Now if you want to restore the snapshot,  you have to export it back an
> > HBase cluster.
> > So on cluster2, you can do:
> > $ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot -D
> > hbase.rootdir=hdfs:///srv2:8082/my-backup-dir -snapshot MySnapshot
> -copy-to
> > hdfs:///hbaseSrv:8082/hbase
> >
> >
> > so, to recap
> >  - You take a snapshot
> >  - You Export the snapshot from HBase Cluster-1 -> to a simple HDFS dir
> in
> > Cluster-2
> >  - Then you want to restore
> >  - You Export the snapshot from HDFS dir in Cluster-2 to HBase Cluster
> (it
> > can be a different one from the original)
> >  - From the hbase shell you can just: clone_snapshot 'snapshotName',
> > 'newTableName' if the table does not exists or use restore_snapshot
> > 'snapshotName', if there's a table with the same name
> >
> >
> > Matteo
> >
> >
> >
> > On Thu, Aug 1, 2013 at 2:54 PM, Siddharth Karandikar <
> > siddharth.karandikar@gmail.com> wrote:
> >
> >> Yeah, thats right. But the issue is, hdfs that I am exporting to is
> >> not under HBase.
> >> Can you please provide some example command to do this...
> >>
> >>
> >> Thanks,
> >> Siddharth
> >>
> >> On Thu, Aug 1, 2013 at 7:17 PM, Matteo Bertozzi <
> theo.bertozzi@gmail.com>
> >> wrote:
> >> > Yes, the export an HDFS path.
> >> > $ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
> >> > -snapshot MySnapshot -copy-to hdfs:///srv2:8082/hbase
> >> >
> >> > so you can export to some /my-backup-dir on your HDFS
> >> > and then you've to export back to an hbase cluster, when you want to
> >> > restore it
> >> >
> >> > Matteo
> >> >
> >> >
> >> >
> >> > On Thu, Aug 1, 2013 at 2:45 PM, Siddharth Karandikar <
> >> > siddharth.karandikar@gmail.com> wrote:
> >> >
> >> >> Can't I export it to plain HDFS? I think that would be very useful.
> >> >>
> >> >> On Thu, Aug 1, 2013 at 7:08 PM, Matteo Bertozzi <
> >> theo.bertozzi@gmail.com>
> >> >> wrote:
> >> >> > The ExportSnapshot will export the snapshot data+metadata, in
> theory,
> >> to
> >> >> > another hbase cluster.
> >> >> > so on the second cluster you'll now be able to do "list_snapshots"
> >> from
> >> >> > shell and see the exported snapshot.
> >> >> > now you can simply do clone_snapshot "snapshot_name",
> "new_table_name"
> >> >> and
> >> >> > you're restoring a snapshot on the second cluster
> >> >> >
> >> >> > assuming that you have removed the snapshot from cluster1 and
you
> >> want to
> >> >> > export back your snapshot..
> >> >> > you just use ExportSnapshot again to move the snapshot from
> cluster2
> >> to
> >> >> > cluster1
> >> >> > and same as before you do a clone_snapshot to restore it
> >> >> >
> >> >> > Matteo
> >> >> >
> >> >> >
> >> >> >
> >> >> > On Thu, Aug 1, 2013 at 2:35 PM, Siddharth Karandikar <
> >> >> > siddharth.karandikar@gmail.com> wrote:
> >> >> >
> >> >> >> Hi there,
> >> >> >>
> >> >> >> I am testing out newly added snapshot capability, ExportSnapshot
> in
> >> >> >> particular.
> >> >> >> Its working fine for me. I am able to run ExportSnapshot properly.
> >> >> >>
> >> >> >> But the biggest (noob) issue is, once exported, is there any
way
> to
> >> >> >> import those snapshots back in hbase? I don't see any
> ImportSnapshot
> >> >> >> util there.
> >> >> >>
> >> >> >>
> >> >> >> Thanks,
> >> >> >> Siddharth
> >> >> >>
> >> >>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message