hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: snapshotexport to outside of cluster
Date Sun, 01 Dec 2013 08:55:39 GMT
Since the remote server doesn't have hdfs, what would the backup be used for ?

Cheers

On Nov 30, 2013, at 10:34 PM, oc tsdb <oc.tsdb@gmail.com> wrote:

> Hi,
> 
>>> Can you clarify what 'without copying to local file system' means ?
> 
> We want to take HBase tables backup.So we periodically take snapshots and
> export data to backup server which is not in cluster.
> 
> Currently we are achieving this as mentioned below.
> 
> 1.Create snapshots from HBase command line
>   lets say we have created snapshot for a table called 'hbase_tabel' and
> its snapshot name is 'hbase_tabel_snapshot'
> 2. Then export snapshot data to remote backup server which is not in
> cluster.We are achieving this in two steps.
>   a. Export snapshot to local file system(sample path /tmp/hbase_backup)
> from hdfs file system as mentioned below
>       hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
> hbase_tabel_snapshot -copy-to file:///tmp/hbase_backup -
>       mappers      16
>      Here we should have enough disc space in local file system to export
> snapshot from hdfs.
>    b.Transfer(scp) exported snapshot from local file system
> (/tmp/hbase_backup)
> to backup server which is not hadoop/HBase running
>       server.
> 
>  That means first we have to export snapshot to local system from hdfs and
> then transfer ( scp ) to remote server.Instead of doing in two steps can we
> directly export snapshot to remote server(which is not in cluster) like how
> it is exporting to another cluster.
> .
> Thanks
> OC
> 
> 
> 
> 
> 
> On Sat, Nov 30, 2013 at 11:11 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> 
>> HBASE-4618 is not related to snapshots.
>> Lars did some work in HBASE-5509, a sub-task of HBASE-4618.
>> 
>> FYI
>> 
>> 
>> On Fri, Nov 29, 2013 at 9:31 PM, Yu Li <carp84@gmail.com> wrote:
>> 
>>> I guess this might be a use case of
>>> HBASE-7912<https://issues.apache.org/jira/browse/HBASE-7912>or
>>> HBASE-4618 <https://issues.apache.org/jira/browse/HBASE-4618>? Maybe
>> some
>>> backup/restore tools taking use of snapshot would be nice to have?
>>> 
>>> 
>>> On 30 November 2013 12:58, Ted Yu <yuzhihong@gmail.com> wrote:
>>> 
>>>> The remote server is in different cluster, right ?
>>>> 
>>>> Looking at ExportSnapshot#ExportMapper#copyFile(), it directly copies
>>> data
>>>> to outputPath which is in the other cluster.
>>>> 
>>>> Can you clarify what 'without copying to local file system' means ?
>>>> 
>>>> Cheers
>>>> 
>>>> 
>>>> On Fri, Nov 29, 2013 at 8:49 PM, oc tsdb <oc.tsdb@gmail.com> wrote:
>>>> 
>>>>> Hi,
>>>>> thanks for your replay.
>>>>> 
>>>>> We are able to export snapshot to another cluster But this is not
>> what
>>> we
>>>>> are looking for.
>>>>> We want to export snapshot to remote server without copying to local
>>> file
>>>>> system.
>>>>> 
>>>>> Thanks
>>>>> OC
>>>>> 
>>>>> On Fri, Nov 29, 2013 at 8:52 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>>>> 
>>>>>> Here is a sample command exporting to another cluster:
>>>>>> 
>>>>>> hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
>>>>>> gs_1383068263 -copy-to hdfs://experiment-1d:8020/hbase -mappers 6
>>>>>> 
>>>>>> 
>>>>>> On Fri, Nov 29, 2013 at 4:02 AM, oc tsdb <oc.tsdb@gmail.com>
>> wrote:
>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> can someone please let us know if we can export snapshots data
>>>> directly
>>>>>> to
>>>>>>> outside of cluster without copying to local.
>>>>>>> 
>>>>>>> Please refer us if there are any blog with example.
>>>>>>> 
>>>>>>> Below command is copying data to local file system under /tmp.
>>>>>>> hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
>>>> mytable
>>>>>>> -copy-to file:///tmp/hbase_backup -mappers 16
>>>>>>> 
>>>>>>> Without copying to local system, we want to export snapshot(eg
>> scp)
>>>>> data
>>>>>> to
>>>>>>> a server which is not part of any cluster but its reachable from
>>>>> cluster.
>>>>>>> 
>>>>>>> Thanks
>>>>>>> OC
>>> 
>>> 
>>> 
>>> --
>>> Best Regards,
>>> Li Yu
>> 

Mime
View raw message