hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dima Spivak <dspi...@cloudera.com>
Subject Re: hbase 'transparent encryption' feature is production ready or not?
Date Mon, 06 Jun 2016 16:16:25 GMT
FWIW, some engineers at Cloudera who worked on adding encryption at rest to
HDFS wrote a blog post on this where they describe negligible performance
impacts on write and only a slight performance degradation on large reads (
http://blog.cloudera.com/blog/2015/01/new-in-cdh-5-3-transparent-encryption-in-hdfs/).
Obviously, your mileage may vary, but in my internal testing, I can also
say I haven't seen much (if any) impact with encryption zones enabled.

-Dima

On Monday, June 6, 2016, Liu, Ming (Ming) <ming.liu@esgyn.cn> wrote:

> Hi, Andrew again,
>
> I still have a question that if we move the encryption to HDFS level, we
> no longer can enable encryption per table I think?
> I assume encryption will impact performance to some extent, so we may
> would like to enable it per table. Is there any performance tests that
> shows how much overhead encryption can introduce? If very small, then I am
> very happy to do it in HDFS and encrypt all data.
> I still not start to study HSM, but if for example, we can setup a
> separate storage, like a NFS, which can be mounted to each node of HBase
> cluster, and we put the key there, is it an acceptable plan?
>
> Thanks,
> Ming
>
> -----邮件原件-----
> 发件人: Andrew Purtell [mailto:apurtell@apache.org <javascript:;>]
> 发送时间: 2016年6月3日 12:27
> 收件人: user@hbase.apache.org <javascript:;>
> 抄送: Zhang, Yi (Eason) <yi.zhang@esgyn.cn <javascript:;>>
> 主题: Re: 答复: hbase 'transparent encryption' feature is production ready or
> not?
>
> > We are now confident to use this feature.
>
> You should test carefully for your use case in any case.
>
> > HSM is a good option, I am new to it. But will look at it.
>
> I recommend using HDFS's transparent encryption feature instead of HBase
> transparent encryption if you're only just now thinking about HSMs and key
> protection in general. Storing the master key on the same nodes as the
> encrypted data will defeat protection. This should be offloaded to a
> protected domain. Hadoop ships with a software KMS that, while it has
> limitations, can be set up on a specially secured server and HDFS TDE can
> take advantage of it. (HBase TDE doesn't support the Hadoop KMS.)
>
> Advice offered for what it's worth (smile)
>
>
> On Thu, Jun 2, 2016 at 9:16 PM, Liu, Ming (Ming) <ming.liu@esgyn.cn
> <javascript:;>> wrote:
>
> > Thank you Andrew!
> >
> > What we hear must be rumor :-) We are now confident to use this feature.
> >
> > HSM is a good option, I am new to it. But will look at it.
> >
> > Thanks,
> > Ming
> > -----邮件原件-----
> > 发件人: Andrew Purtell [mailto:apurtell@apache.org <javascript:;>]
> > 发送时间: 2016年6月3日 8:59
> > 收件人: user@hbase.apache.org <javascript:;>
> > 抄送: Zhang, Yi (Eason) <yi.zhang@esgyn.cn <javascript:;>>
> > 主题: Re: hbase 'transparent encryption' feature is production ready or
> not?
> >
> > > We heard from various sources that it is not production ready before.
> >
> > ​Said by whom, specifically? ​
> >
> > ​> During our tests, we do find out it works not very stable, but
> > probably due to our lack of experience of this feature.
> >
> > If you have something repeatable, please consider filing a JIRA to
> > report the problem.
> >
> > > And, we now save the encryption key in the disk, so we were
> > > wondering,
> > this is something not secure.
> >
> > Data keys are encrypted with a master key which must be protected. The
> > out of the box key provider stores the master key in a local keystore.
> > That's not sufficient protection. In a production environment you will
> > want to use a HSM. Most (all?) HSMs support the keystore API. If that
> > is not sufficient, our KeyProvider API is extensible for the solution
> > you choose to employ in production.
> >
> > ​Have you looked at HDFS transparent encryption?
> >
> > https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/
> > TransparentEncryption.html Because it works at the HDFS layer it's a
> > more general solution. Be careful what version of Hadoop you use if
> > opting for HDFS TDE, though. Pick the most recent release. Slightly
> > older versions (like 2.6.0) had fatal bugs if used in conjunction with
> > HBase.
> >
> >
> >
> > On Thu, Jun 2, 2016 at 5:52 PM, Liu, Ming (Ming) <ming.liu@esgyn.cn
> <javascript:;>>
> > wrote:
> >
> > > Hi, all,
> > >
> > > We are trying to deploy the 'transparent encryption' feature of
> > > HBase , described in HBase reference guide:
> > > https://hbase.apache.org/book.html#hbase.encryption.server  , in our
> > > product.
> > > We heard from various sources that it is not production ready before.
> > >
> > > During our tests, we do find out it works not very stable, but
> > > probably due to our lack of experience of this feature. It works
> > > sometime, sometimes not work, and retry the same configuration, it
> > > work again. We were using HBase 1.0.
> > >
> > > Could anyone give us some information that this feature is already
> > > stable and can be used in a production environment?
> > >
> > > And, we now save the encryption key in the disk, so we were
> > > wondering, this is something not secure. Since the key is at same
> > > place with data, anyone can decode the data because if he/she can
> > > access the data, he/she can access the key as well. Is there any
> > > best practice about how to manage the key?
> > >
> > > Thanks,
> > > Ming
> > >
> > >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein (via Tom White)
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message