hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajesh Kartha <karth...@gmail.com>
Subject Re: Encryption At Rest Question
Date Tue, 24 Feb 2015 22:16:32 GMT
Thank you Olivier,

I suppose with the first suggestion - locking the dir to be unreadable for
other users, the HDFS permissions would
kick in and prevent an unwarranted user to read them.
However, I wanted to see the actual encrypted data so I used the second
approach you suggested. With hadoop fsck /mysecureDir -files -blocks
-locations get the blocks for the directory, then go to the data node and
perform a cat to see cryptic data for those block.


On Tue, Feb 24, 2015 at 12:28 PM, Olivier Renault <orenault@hortonworks.com>

>   You can try looking at it with a user who doesn’t have permission to
> the folder. An alternative is to check which block it is on Linux and
> looking at the block using cat from a linux shell.
>  Olivier
>   From: Rajesh Kartha <kartha02@gmail.com>
> Reply-To: "user@hadoop.apache.org" <user@hadoop.apache.org>
> Date: Tuesday, 24 February 2015 19:47
> To: "user@hadoop.apache.org" <user@hadoop.apache.org>
> Cc: "hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>
> Subject: Re: Encryption At Rest Question
>     I was trying out the Transparent data at rest encryption and was able
> to setup the KMS, zones etc. and add
>  files to the zone.
>  How do I confirm if the files I added to the encryption zone are
> encrypted ? Is there a way to view
>  the raw file, a *hdfs fs -cat *shows me the actual contents of the files
> since the datanode decrypts it
>  before sending it.
>  Thanks,
>  Rajesh
> On Fri, Feb 20, 2015 at 11:42 PM, Ranadip Chatterjee <ranadip.c@gmail.com>
> wrote:
>>  In case of SSL enabled cluster, the DEK will be encrypted on the wire
>> by the SSL layer.
>>  In case of non-SSL enabled cluster, it is not. But the intercepter only
>> gets the DEK and not the encrypted data, so the data is still safe. Only if
>> the intercepter also manages to gain access to the encrypted data block and
>> associate that with the corresponding DEK, then the data is compromised.
>> Given that each HDFS file has a different DEK, the intercepter has to gain
>> quite a bit of access before the data is compromised.
>> On 18 February 2015 at 00:04, Plamen Jeliazkov <
>> plamen.jeliazkov@wandisco.com> wrote:
>>> Hey guys,
>>>  I had a question about how the new file encryption work done primarily
>>> in HDFS-6134.
>>>  I was just curious, how is the DEK protected on the wire?
>>> Particularly after the KMS decrypts the EDEK and returns it to the
>>> client.
>>>  Thanks,
>>> -Plamen
>>>  5 reasons your Hadoop needs WANdisco
>>> <http://www.wandisco.com/system/files/documentation/5-Reasons.pdf>
>>> Listed on the London Stock Exchange: WAND
>>> <http://www.bloomberg.com/quote/WAND:LN>
>>> BE PRIVILEGED.  If this message was misdirected, WANdisco, Inc. and its
>>> subsidiaries, ("WANdisco") does not waive any confidentiality or
>>> privilege.  If you are not the intended recipient, please notify us
>>> immediately and destroy the message without disclosing its contents to
>>> anyone.  Any distribution, use or copying of this e-mail or the information
>>> it contains by other than an intended recipient is unauthorized.  The views
>>> and opinions expressed in this e-mail message are the author's own and may
>>> not reflect the views and opinions of WANdisco, unless the author is
>>> authorized by WANdisco to express such views or opinions on its behalf.
>>> All email sent to or from this address is subject to electronic storage and
>>> review by WANdisco.  Although WANdisco operates anti-virus programs, it
>>> does not accept responsibility for any damage whatsoever caused by viruses
>>> being passed.
>> --
>> Regards,
>> Ranadip Chatterjee

View raw message