Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of yuzhihong@gmail.com
 designates 209.85.192.170 as permitted sender)
References: 
 <CAH37JUQTPBtr1g5=aHFPMnH-Of1vLTdmFvfarxciFTxEBBnSgw@mail.gmail.com>
 <CALte62xmJs4W3NXjwPEAyFrmHOxteDVm3tr9NFpZbKtCUTO7iQ@mail.gmail.com>
 <CAH37JUTsXvYbm15=penqk5qSUTN4pwKh6bswd8huci4yhCuJWw@mail.gmail.com>
 <CALte62z66Az0jtyLRwCk4HKKpVA7mg1wZCXMgV=BFyB6Hj6q-w@mail.gmail.com>
 <CAM7-19K4-GY7xUVbk6rWrg2nhZnmBZFct0r16o3uJ4iuwu+P7A@mail.gmail.com>
 <CALte62zb9Xv4J7xPh-ofo=k86zyvq_p5T_Y0yDmH1Qz3jYs2Zg@mail.gmail.com>
 <CAH37JUQyNkQp7frbi9oCjJAFsd-LBhht22AEC=J-1U1sjb3Hbg@mail.gmail.com>
 <898E038B-CA9C-4350-BB7F-2161E1F405CC@gmail.com>
 <CAH37JUQ=rARqg6u3pSazWeKiiy46Wy8pq2OstcdnPZM4PjO1aw@mail.gmail.com>
Mime-Version: 1.0 (1.0)
In-Reply-To: 
 <CAH37JUQ=rARqg6u3pSazWeKiiy46Wy8pq2OstcdnPZM4PjO1aw@mail.gmail.com>
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-Id: <2A3FF253-8CDC-4519-A56E-11C99CA2BAA9@gmail.com>
Cc: "user@hbase.apache.org" <user@hbase.apache.org>
From: Ted Yu <yuzhihong@gmail.com>
Subject: Re: snapshotexport to outside of cluster
Date: Sun, 1 Dec 2013 07:04:59 -0800
To: "user@hbase.apache.org" <user@hbase.apache.org>

What if there is a bad sector on the disk of remote server ?
Using hdfs, you're shielded from such scenario. 

On Dec 1, 2013, at 6:56 AM, oc tsdb <oc.tsdb@gmail.com> wrote:

> We want to use that exported data in case of any disaster happens.
> 
> Thanks
> -OC
> 
> 
> On Sun, Dec 1, 2013 at 2:25 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> 
>> Since the remote server doesn't have hdfs, what would the backup be used
>> for ?
>> 
>> Cheers
>> 
>> On Nov 30, 2013, at 10:34 PM, oc tsdb <oc.tsdb@gmail.com> wrote:
>> 
>>> Hi,
>>> 
>>>>> Can you clarify what 'without copying to local file system' means ?
>>> 
>>> We want to take HBase tables backup.So we periodically take snapshots and
>>> export data to backup server which is not in cluster.
>>> 
>>> Currently we are achieving this as mentioned below.
>>> 
>>> 1.Create snapshots from HBase command line
>>>  lets say we have created snapshot for a table called 'hbase_tabel' and
>>> its snapshot name is 'hbase_tabel_snapshot'
>>> 2. Then export snapshot data to remote backup server which is not in
>>> cluster.We are achieving this in two steps.
>>>  a. Export snapshot to local file system(sample path /tmp/hbase_backup)
>>> from hdfs file system as mentioned below
>>>      hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
>>> hbase_tabel_snapshot -copy-to file:///tmp/hbase_backup -
>>>      mappers      16
>>>     Here we should have enough disc space in local file system to export
>>> snapshot from hdfs.
>>>   b.Transfer(scp) exported snapshot from local file system
>>> (/tmp/hbase_backup)
>>> to backup server which is not hadoop/HBase running
>>>      server.
>>> 
>>> That means first we have to export snapshot to local system from hdfs
>> and
>>> then transfer ( scp ) to remote server.Instead of doing in two steps can
>> we
>>> directly export snapshot to remote server(which is not in cluster) like
>> how
>>> it is exporting to another cluster.
>>> .
>>> Thanks
>>> OC
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On Sat, Nov 30, 2013 at 11:11 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>> 
>>>> HBASE-4618 is not related to snapshots.
>>>> Lars did some work in HBASE-5509, a sub-task of HBASE-4618.
>>>> 
>>>> FYI
>>>> 
>>>> 
>>>> On Fri, Nov 29, 2013 at 9:31 PM, Yu Li <carp84@gmail.com> wrote:
>>>> 
>>>>> I guess this might be a use case of
>>>>> HBASE-7912<https://issues.apache.org/jira/browse/HBASE-7912>or
>>>>> HBASE-4618 <https://issues.apache.org/jira/browse/HBASE-4618>? Maybe
>>>> some
>>>>> backup/restore tools taking use of snapshot would be nice to have?
>>>>> 
>>>>> 
>>>>> On 30 November 2013 12:58, Ted Yu <yuzhihong@gmail.com> wrote:
>>>>> 
>>>>>> The remote server is in different cluster, right ?
>>>>>> 
>>>>>> Looking at ExportSnapshot#ExportMapper#copyFile(), it directly copies
>>>>> data
>>>>>> to outputPath which is in the other cluster.
>>>>>> 
>>>>>> Can you clarify what 'without copying to local file system' means ?
>>>>>> 
>>>>>> Cheers
>>>>>> 
>>>>>> 
>>>>>> On Fri, Nov 29, 2013 at 8:49 PM, oc tsdb <oc.tsdb@gmail.com> wrote:
>>>>>> 
>>>>>>> Hi,
>>>>>>> thanks for your replay.
>>>>>>> 
>>>>>>> We are able to export snapshot to another cluster But this is not
>>>> what
>>>>> we
>>>>>>> are looking for.
>>>>>>> We want to export snapshot to remote server without copying to local
>>>>> file
>>>>>>> system.
>>>>>>> 
>>>>>>> Thanks
>>>>>>> OC
>>>>>>> 
>>>>>>> On Fri, Nov 29, 2013 at 8:52 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>>>>>> 
>>>>>>>> Here is a sample command exporting to another cluster:
>>>>>>>> 
>>>>>>>> hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
>>>>>>>> gs_1383068263 -copy-to hdfs://experiment-1d:8020/hbase -mappers 6
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Fri, Nov 29, 2013 at 4:02 AM, oc tsdb <oc.tsdb@gmail.com>
>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> can someone please let us know if we can export snapshots data
>>>>>> directly
>>>>>>>> to
>>>>>>>>> outside of cluster without copying to local.
>>>>>>>>> 
>>>>>>>>> Please refer us if there are any blog with example.
>>>>>>>>> 
>>>>>>>>> Below command is copying data to local file system under /tmp.
>>>>>>>>> hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
>>>>>> mytable
>>>>>>>>> -copy-to file:///tmp/hbase_backup -mappers 16
>>>>>>>>> 
>>>>>>>>> Without copying to local system, we want to export snapshot(eg
>>>> scp)
>>>>>>> data
>>>>>>>> to
>>>>>>>>> a server which is not part of any cluster but its reachable from
>>>>>>> cluster.
>>>>>>>>> 
>>>>>>>>> Thanks
>>>>>>>>> OC
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Best Regards,
>>>>> Li Yu
>>