Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Subject: Re: HBase Encryption - HDFS Vs HBase Level
To: user@hbase.apache.org
References: <CAFh5nhxWke2XvUr70t_nxk1nVkJyE8G8BmV7c4ms+LzAs5WagA@mail.gmail.com>
 <CALte62y3ES3XRtenNeopgZhOB-m0vvs7QaiMbjX_cPV3CF6DaQ@mail.gmail.com>
From: Josh Elser <elserj@apache.org>
Message-ID: <d1f00c1f-28c1-b294-db10-95bbd1a1c174@apache.org>
Date: Fri, 18 Aug 2017 13:55:39 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:56.0)
 Gecko/20100101 Thunderbird/56.0
MIME-Version: 1.0
In-Reply-To: <CALte62y3ES3XRtenNeopgZhOB-m0vvs7QaiMbjX_cPV3CF6DaQ@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
archived-at: Fri, 18 Aug 2017 17:55:44 -0000

Some specificity (as I still remember it too vividly)

https://issues.apache.org/jira/browse/HADOOP-11710

Our Sean got this one fixed for 2.6.1, and would by why using HDFS 
transparent encryption with 2.6.0 will flat-out not work :)

On 8/18/17 1:35 PM, Ted Yu wrote:
> Please see the 'Hadoop 2.6.x' bullet under
> http://hbase.apache.org/book.html#hadoop
> 
> FYI
> 
> On Fri, Aug 18, 2017 at 10:25 AM, Saad Mufti <saad.mufti@gmail.com> wrote:
> 
>> Hi,
>>
>> I'm looking for some guidance as our security team is requiring us to
>> implement encryption of our HBase data at rest and in motion. I'm reading
>> the docs and doing research and the choice seems to be between doing it at
>> the HBase level or the more general HDFS level.
>>
>> I am leaning towards HDFS level as there is some other data that is derived
>> from HBase in HDFS and it would be nice to have that encrypted as well.
>> Once set up the encryption is supposed to transparent to clients. We're
>> still at HBase 1.0 level, we're using a Cloudera 5.5 based distribution but
>> no commercial license. For reasons I won't go into upgrading is not an
>> option in the short term and we need to implement encryption before that
>>
>> But I have a warning in a google groups somewhere (can't find it anymore)
>> that warns that HDFS level encryption doesn't play well with HBase if on
>> Hadoop 2.6.x, which we're at. Does anyone know the specific issue, or if
>> there is a specific ticket I can look at to see if our Hadoop distro
>> includes that fix?
>>
>> Also, out of the box the Key Management Server included in Hadoop is based
>> on a simple file based Java Keystore and there are warnings that it is not
>> suitable for production environments. Cloudera has their own proprietary
>> KMS but we don't have a license to it. Can anyone share what groups that
>> use pure open source distros are using as their KMS when implementing
>> encryption in production environments?
>>
>> Thanks in advance for any guidance you can provide.
>>
>> ----
>> Saad
>>
>