Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9ADA617502 for ; Tue, 24 Feb 2015 20:29:40 +0000 (UTC) Received: (qmail 40909 invoked by uid 500); 24 Feb 2015 20:29:18 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 40775 invoked by uid 500); 24 Feb 2015 20:29:18 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 40687 invoked by uid 99); 24 Feb 2015 20:29:18 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 24 Feb 2015 20:29:18 +0000 X-ASF-Spam-Status: No, hits=4.2 required=5.0 tests=FSL_HELO_BARE_IP_2,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of orenault@hortonworks.com designates 64.78.52.184 as permitted sender) Received: from [64.78.52.184] (HELO relayvx11b.securemail.intermedia.net) (64.78.52.184) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 24 Feb 2015 20:29:13 +0000 Received: from securemail.intermedia.net (localhost [127.0.0.1]) by emg-ca-1-1.localdomain (Postfix) with ESMTP id 255B953DB0 for ; Tue, 24 Feb 2015 12:28:11 -0800 (PST) Subject: Re: Encryption At Rest Question MIME-Version: 1.0 x-echoworx-emg-received: Tue, 24 Feb 2015 12:28:11.142 -0800 x-echoworx-msg-id: 00dcf2fd-9c87-46c7-924e-a2a9a0dafaa9 x-echoworx-action: delivered Received: from 10.254.155.14 ([10.254.155.14]) by emg-ca-1-1 (JAMES SMTP Server 2.3.2) with SMTP ID 264 for ; Tue, 24 Feb 2015 12:28:11 -0800 (PST) Received: from MBX080-W4-CO-1.exch080.serverpod.net (unknown [10.224.117.101]) by emg-ca-1-1.localdomain (Postfix) with ESMTP id E203153E1D for ; Tue, 24 Feb 2015 12:28:10 -0800 (PST) Received: from MBX080-W4-CO-1.exch080.serverpod.net (10.224.117.101) by MBX080-W4-CO-1.exch080.serverpod.net (10.224.117.101) with Microsoft SMTP Server (TLS) id 15.0.1044.25; Tue, 24 Feb 2015 12:28:10 -0800 Received: from MBX080-W4-CO-1.exch080.serverpod.net ([10.224.117.101]) by mbx080-w4-co-1.exch080.serverpod.net ([10.224.117.101]) with mapi id 15.00.1044.021; Tue, 24 Feb 2015 12:28:10 -0800 From: Olivier Renault To: "user@hadoop.apache.org" Thread-Topic: Encryption At Rest Question Thread-Index: AQHQSxSKTeu8Snhq1E2S6sczrfHP6pz7Q8OAgAWBq4CAAAtPAA== Date: Tue, 24 Feb 2015 20:28:09 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [86.184.130.193] x-source-routing-agent: Processed Content-Type: multipart/alternative; boundary="_000_D1128EFB1D3DBorenaulthortonworkscom_" X-Virus-Checked: Checked by ClamAV on apache.org --_000_D1128EFB1D3DBorenaulthortonworkscom_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable You can try looking at it with a user who doesn't have permission to the fo= lder. An alternative is to check which block it is on Linux and looking at = the block using cat from a linux shell. Olivier From: Rajesh Kartha > Reply-To: "user@hadoop.apache.org" > Date: Tuesday, 24 February 2015 19:47 To: "user@hadoop.apache.org" > Cc: "hdfs-dev@hadoop.apache.org" > Subject: Re: Encryption At Rest Question I was trying out the Transparent data at rest encryption and was able to se= tup the KMS, zones etc. and add files to the zone. How do I confirm if the files I added to the encryption zone are encrypted = ? Is there a way to view the raw file, a hdfs fs -cat shows me the actual contents of the files sinc= e the datanode decrypts it before sending it. Thanks, Rajesh On Fri, Feb 20, 2015 at 11:42 PM, Ranadip Chatterjee > wrote: In case of SSL enabled cluster, the DEK will be encrypted on the wire by th= e SSL layer. In case of non-SSL enabled cluster, it is not. But the intercepter only get= s the DEK and not the encrypted data, so the data is still safe. Only if th= e intercepter also manages to gain access to the encrypted data block and a= ssociate that with the corresponding DEK, then the data is compromised. Giv= en that each HDFS file has a different DEK, the intercepter has to gain qui= te a bit of access before the data is compromised. On 18 February 2015 at 00:04, Plamen Jeliazkov > wrote: Hey guys, I had a question about how the new file encryption work done primarily in H= DFS-6134. I was just curious, how is the DEK protected on the wire? Particularly after the KMS decrypts the EDEK and returns it to the client. Thanks, -Plamen 5 reasons your Hadoop needs WANdisco Listed on the London Stock Exchange: WAND THIS MESSAGE AND ANY ATTACHMENTS ARE CONFIDENTIAL, PROPRIETARY, AND MAY BE = PRIVILEGED. If this message was misdirected, WANdisco, Inc. and its subsid= iaries, ("WANdisco") does not waive any confidentiality or privilege. If y= ou are not the intended recipient, please notify us immediately and destroy= the message without disclosing its contents to anyone. Any distribution, = use or copying of this e-mail or the information it contains by other than = an intended recipient is unauthorized. The views and opinions expressed in= this e-mail message are the author's own and may not reflect the views and= opinions of WANdisco, unless the author is authorized by WANdisco to expre= ss such views or opinions on its behalf. All email sent to or from this ad= dress is subject to electronic storage and review by WANdisco. Although WA= Ndisco operates anti-virus programs, it does not accept responsibility for = any damage whatsoever caused by viruses being passed. -- Regards, Ranadip Chatterjee --_000_D1128EFB1D3DBorenaulthortonworkscom_ Content-Type: text/html; charset="iso-8859-1" Content-ID: Content-Transfer-Encoding: quoted-printable
You can try looking at it with a user who doesn’t have permissio= n to the folder. An alternative is to check which block it is on Linux and = looking at the block using cat from a linux shell. 

Olivier


From: Rajesh Kartha <kartha02@gmail.com>
Reply-To: "user@hadoop.apache.org" <user@hadoop.apache.org>
Date: Tuesday, 24 February 2015 19:= 47
To: "user@hadoop.apache.org" <user@hadoop.apache.org>
Cc: "hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>
Subject: Re: Encryption At Rest Que= stion

I was trying out the Transparent data at rest encryption and was able = to setup the KMS, zones etc. and add
files to the zone.

How do I confirm if the files I added to the encryption zone are encrypted = ? Is there a way to view
the raw file, a hdfs fs -cat shows me the actual contents of the fil= es since the datanode decrypts it
before sending it.

Thanks,
Rajesh


On Fri, Feb 20, 2015 at 11:42 PM, Ranadip Chatte= rjee <ranadip.c@gmai= l.com> wrote:
In case of SSL enabled cluster, the DEK will be encrypted on the wire = by the SSL layer.

In case of non-SSL enabled cluster, it is not. But the intercepter only get= s the DEK and not the encrypted data, so the data is still safe. Only if th= e intercepter also manages to gain access to the encrypted data block and a= ssociate that with the corresponding DEK, then the data is compromised. Given that each HDFS file has a differe= nt DEK, the intercepter has to gain quite a bit of access before the data i= s compromised.

On 18 February 2015 at 00:04, Plamen Jeliazkov <= span dir=3D"ltr"> <plam= en.jeliazkov@wandisco.com> wrote:
Hey guys,

I had a question about how the new file encryption work done primarily= in HDFS-6134.

I was just curious, how is the DEK protected on the wire? 
Particularly after the KMS decrypts the EDEK and returns it to the cli= ent.

Thanks,
-Plamen



5 reasons your Hadoop needs WANdisco

Listed on the London Stock Exchange: WAND

THIS ME= SSAGE AND ANY ATTACHMENTS ARE CONFIDENTIAL, PROPRIETARY, AND MAY BE PRIVILE= GED.  If this message was misdirected, WANdisco, Inc. and its subsidia= ries, ("WANdisco") does not waive any confidentiality or privilege.  If you are not the intended recipient, please notify u= s immediately and destroy the message without disclosing its contents to an= yone.  Any distribution, use or copying of this e-mail or the informat= ion it contains by other than an intended recipient is unauthorized.  The views and opinions expressed in this = e-mail message are the author's own and may not reflect the views and opini= ons of WANdisco, unless the author is authorized by WANdisco to express suc= h views or opinions on its behalf.  All email sent to or from this address is subject to electronic storage and re= view by WANdisco.  Although WANdisco operates anti-virus programs, it = does not accept responsibility for any damage whatsoever caused by viruses = being passed.




--
Regards,
Ranadip Chatterjee

 --_000_D1128EFB1D3DBorenaulthortonworkscom_--