hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From oc tsdb <oc.t...@gmail.com>
Subject Re: snapshotexport to outside of cluster
Date Sun, 01 Dec 2013 06:34:10 GMT
Hi,

>>Can you clarify what 'without copying to local file system' means ?

We want to take HBase tables backup.So we periodically take snapshots and
export data to backup server which is not in cluster.

Currently we are achieving this as mentioned below.

1.Create snapshots from HBase command line
   lets say we have created snapshot for a table called 'hbase_tabel' and
its snapshot name is 'hbase_tabel_snapshot'
2. Then export snapshot data to remote backup server which is not in
cluster.We are achieving this in two steps.
   a. Export snapshot to local file system(sample path /tmp/hbase_backup)
from hdfs file system as mentioned below
       hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
hbase_tabel_snapshot -copy-to file:///tmp/hbase_backup -
       mappers      16
      Here we should have enough disc space in local file system to export
snapshot from hdfs.
    b.Transfer(scp) exported snapshot from local file system
(/tmp/hbase_backup)
to backup server which is not hadoop/HBase running
       server.

  That means first we have to export snapshot to local system from hdfs and
then transfer ( scp ) to remote server.Instead of doing in two steps can we
directly export snapshot to remote server(which is not in cluster) like how
it is exporting to another cluster.
.
Thanks
OC





On Sat, Nov 30, 2013 at 11:11 AM, Ted Yu <yuzhihong@gmail.com> wrote:

> HBASE-4618 is not related to snapshots.
> Lars did some work in HBASE-5509, a sub-task of HBASE-4618.
>
> FYI
>
>
> On Fri, Nov 29, 2013 at 9:31 PM, Yu Li <carp84@gmail.com> wrote:
>
> > I guess this might be a use case of
> > HBASE-7912<https://issues.apache.org/jira/browse/HBASE-7912>or
> > HBASE-4618 <https://issues.apache.org/jira/browse/HBASE-4618>? Maybe
> some
> > backup/restore tools taking use of snapshot would be nice to have?
> >
> >
> > On 30 November 2013 12:58, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> > > The remote server is in different cluster, right ?
> > >
> > > Looking at ExportSnapshot#ExportMapper#copyFile(), it directly copies
> > data
> > > to outputPath which is in the other cluster.
> > >
> > > Can you clarify what 'without copying to local file system' means ?
> > >
> > > Cheers
> > >
> > >
> > > On Fri, Nov 29, 2013 at 8:49 PM, oc tsdb <oc.tsdb@gmail.com> wrote:
> > >
> > > > Hi,
> > > > thanks for your replay.
> > > >
> > > > We are able to export snapshot to another cluster But this is not
> what
> > we
> > > > are looking for.
> > > > We want to export snapshot to remote server without copying to local
> > file
> > > > system.
> > > >
> > > > Thanks
> > > > OC
> > > >
> > > > On Fri, Nov 29, 2013 at 8:52 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> > > >
> > > > > Here is a sample command exporting to another cluster:
> > > > >
> > > > > hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
> > > > > gs_1383068263 -copy-to hdfs://experiment-1d:8020/hbase -mappers 6
> > > > >
> > > > >
> > > > > On Fri, Nov 29, 2013 at 4:02 AM, oc tsdb <oc.tsdb@gmail.com>
> wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > can someone please let us know if we can export snapshots data
> > > directly
> > > > > to
> > > > > > outside of cluster without copying to local.
> > > > > >
> > > > > > Please refer us if there are any blog with example.
> > > > > >
> > > > > > Below command is copying data to local file system under /tmp.
> > > > > > hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
> > > mytable
> > > > > > -copy-to file:///tmp/hbase_backup -mappers 16
> > > > > >
> > > > > > Without copying to local system, we want to export snapshot(eg
> scp)
> > > > data
> > > > > to
> > > > > > a server which is not part of any cluster but its reachable
from
> > > > cluster.
> > > > > >
> > > > > > Thanks
> > > > > > OC
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Best Regards,
> > Li Yu
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message