hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akmal Abbasov <akmal.abba...@icloud.com>
Subject Re: hbase security issue
Date Fri, 13 Mar 2015 07:14:15 GMT

Hi Talat,
I was considering replication, but decided to start with snapshots. Moreover, there are some
drawbacks with replication, like propagation of user error, etc.
Also I need a secure connection between data-centers, and I can't find information about this.


> On 13 Mar 2015, at 05:45, Talat Uyarer <talat@uyarer.com> wrote:
> 
> Hi Akmal,
> 
> Why do not you use Cluster Replication ?
> 
> [1]  http://hbase.apache.org/book.html#_cluster_replication
> On Mar 12, 2015 11:40 PM, "Vladimir Rodionov" <vladrodionov@gmail.com>
> wrote:
> 
>> Thanks, Jerry. I think webdfs is preferable as since it is natively
>> supported by hdfs (name node and data nodes) and traffic does not pass
>> single gateway?
>> 
>> Found this link how to set up webdfs over ssl:
>> 
>> http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.7/bk_Security_Guide/content/ch_wire-webhdfs-mr-yarn.html
>> 
>> Cool. If works :).
>> 
>> -Vlad
>> 
>> 
>>> On Thu, Mar 12, 2015 at 2:24 PM, Jerry He <jerryjch@gmail.com> wrote:
>>> 
>>> Hi, Vladimir
>>> 
>>> Hope I understand your question correctly.
>>> If both local cluster and remote cluster are Kerberos enabled,
>>> ExportSnapshot from local to remote will work as long as both
>>> clusters' Kerberos
>>> have been set up in a way that they understand each other.
>>> If the remote cluster's httpfs/webhdfs port is protected by https
>> security,
>>> after you set up the certificate on the client side, you will be able to
>>> talk to the remote port with SSL protection.
>>> 
>>> Jerry
>>> 
>>> 
>>> On Thu, Mar 12, 2015 at 1:48 PM, Vladimir Rodionov <
>> vladrodionov@gmail.com
>>> wrote:
>>> 
>>>>>> You can also specify the remote target with a httpfs or webfdfs url,
>>>> which
>>>>>> then you can leverage SSL on the transport.
>>>> 
>>>> What if remote cluster has security enabled? Will it work?
>>>> 
>>>> -Vlad
>>>> 
>>>>> On Thu, Mar 12, 2015 at 1:39 PM, Jerry He <jerryjch@gmail.com>
wrote:
>>>>> 
>>>>> ExportSnapshot does not use DistCp but directly use FileSystem API to
>>>> copy,
>>>>> as Vladimir mentioned.
>>>>> But ExportSnapshot supports exporting to a remote target cluster.
>> Give
>>>> the
>>>>> full hdfs url.
>>>>> You can also specify the remote target with a httpfs or webfdfs url,
>>>> which
>>>>> then you can leverage SSL on the transport.
>>>>> 
>>>>> You also can copy to local cluster and use DistCp to copy to remote
>>>>> cluster.
>>>>> 
>>>>> Jerry
>>>>> 
>>>>> On Thu, Mar 12, 2015 at 12:28 PM, Vladimir Rodionov <
>>>>> vladrodionov@gmail.com>
>>>>> wrote:
>>>>> 
>>>>>> No, ExportSnapshot does not use DistCp it runs its own M/R job to
>>> copy
>>>>> data
>>>>>> over to a new destination.
>>>>>> 
>>>>>> In a map task it uses HDFS API to create/write data to a new
>>>> destination.
>>>>>> Therefore, the easiest way to secure communication
>>>>>> during this operation is to use secure HDFS transport.
>> http://www.cloudera.com/content/cloudera/en/documentation/cdh4/v4-3-1/CDH4-Security-Guide/cdh4sg_topic_14_2.html
>>>>>> 
>>>>>> but there is caveat ...
>>>>>> 
>>>>>> ExportSnapshot does not support external cluster configuration -
>> you
>>>>> can't
>>>>>> provide path to external cluster config dir. This seems like a good
>>>>> feature
>>>>>> request.
>>>>>> 
>>>>>> -Vlad
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Thu, Mar 12, 2015 at 10:38 AM, Akmal Abbasov <
>>>>> akmal.abbasov@icloud.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> Hi, I am new to Hadoop Hbase. I have a Hbase cluster in one
>>>> datacenter,
>>>>>>> and I need to  create a backup in the second one. Currently the
>>>> second
>>>>>>> HBase cluster is ready, and I would like to import data from
>> first
>>>>>> cluster.
>>>>>>> I would like to use exportSnapshot tool for this, I’ve tried
it
>> one
>>>> my
>>>>>>> test environment, and it worked well.
>>>>>>> But, since know I am going to export to a different cluster in
>>>>> different
>>>>>>> datacenter, I would like to be sure that my data is secure. So
>> how
>>> I
>>>>> can
>>>>>>> make exportSnapshot secure?
>>>>>>> As far as I understood exportSnapshot uses distcp tool to copy
>>>> snapshot
>>>>>> to
>>>>>>> destination cluster, so in this case is it enough to configure
>>>> distcp?
>>>>>>> Thank you!
>> 

Mime
View raw message