hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: HBase Encryption - HDFS Vs HBase Level
Date Fri, 18 Aug 2017 19:20:48 GMT
The Hadoop KMS in 2.6 or 2.7 can be suitable for demos or prototypes but I
would advise against using it for more than that. Recently the KMS has seen
a number of security improvements. Because it is fairly self contained, you
can check out branch-2.8 or branch-2, build everything, extract the KMS,
and use that.

For what it is worth at my employer we are considering HDFS at rest
encryption. We are building our own key management infrastructure,
incorporating various security and business requirements, and will
implement to the KMS on-wire API for providing key management services to

On Fri, Aug 18, 2017 at 10:25 AM, Saad Mufti <saad.mufti@gmail.com> wrote:

> Hi,
> I'm looking for some guidance as our security team is requiring us to
> implement encryption of our HBase data at rest and in motion. I'm reading
> the docs and doing research and the choice seems to be between doing it at
> the HBase level or the more general HDFS level.
> I am leaning towards HDFS level as there is some other data that is derived
> from HBase in HDFS and it would be nice to have that encrypted as well.
> Once set up the encryption is supposed to transparent to clients. We're
> still at HBase 1.0 level, we're using a Cloudera 5.5 based distribution but
> no commercial license. For reasons I won't go into upgrading is not an
> option in the short term and we need to implement encryption before that
> But I have a warning in a google groups somewhere (can't find it anymore)
> that warns that HDFS level encryption doesn't play well with HBase if on
> Hadoop 2.6.x, which we're at. Does anyone know the specific issue, or if
> there is a specific ticket I can look at to see if our Hadoop distro
> includes that fix?
> Also, out of the box the Key Management Server included in Hadoop is based
> on a simple file based Java Keystore and there are warnings that it is not
> suitable for production environments. Cloudera has their own proprietary
> KMS but we don't have a license to it. Can anyone share what groups that
> use pure open source distros are using as their KMS when implementing
> encryption in production environments?
> Thanks in advance for any guidance you can provide.
> ----
> Saad

Best regards,

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message