Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A74EEED57 for ; Thu, 28 Feb 2013 02:07:22 +0000 (UTC) Received: (qmail 15342 invoked by uid 500); 28 Feb 2013 02:07:14 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 15166 invoked by uid 500); 28 Feb 2013 02:07:14 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 14747 invoked by uid 99); 28 Feb 2013 02:07:14 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Feb 2013 02:07:14 +0000 X-ASF-Spam-Status: No, hits=2.9 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [212.82.109.207] (HELO nm25-vm6.bullet.mail.ird.yahoo.com) (212.82.109.207) by apache.org (qpsmtpd/0.29) with SMTP; Thu, 28 Feb 2013 02:07:08 +0000 Received: from [77.238.189.233] by nm25.bullet.mail.ird.yahoo.com with NNFMP; 28 Feb 2013 02:06:46 -0000 Received: from [212.82.108.118] by tm14.bullet.mail.ird.yahoo.com with NNFMP; 28 Feb 2013 02:06:46 -0000 Received: from [127.0.0.1] by omp1027.mail.ird.yahoo.com with NNFMP; 28 Feb 2013 02:06:46 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 220255.43110.bm@omp1027.mail.ird.yahoo.com Received: (qmail 96411 invoked by uid 60001); 28 Feb 2013 02:00:06 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.fr; s=s1024; t=1362016806; bh=f2/36OWLTquLu7XzpU8/kmeTzgMb/IkLOd2feY67gus=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=wD7k6+xzASv+RqDggVrF3H6N6CE2O41++MKjR0lvdD5Agx8rmmtHAI2vkP0IflkMlo9iEPtzWNZkzPcx6Ja6fCXQ5CwaTaVgEXoWhxq4A16cqYAn+7zUjK5HgV+w+gqsAaPs5+5JABQvij6GTChVYUpil477hVhOoO+GMXQHVB0= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.fr; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=qIwbgxOsbvTiNUWAyhpAblhRWQpwcj9cuiKEkRTw7/LtCpPNXJbPA4lkMkvMvekHyTqzWsZRXweTAgtjYxZ76OZy0TxSxLjYL6mku9MmAO9DEi9CKUXSkE+BBcJ8xd4/w2D1xZne1o1wtX6/7enijV3F2xV4+YzW2O3pZQXTInA=; X-YMail-OSG: PKObWOUVM1lcHfWdXwzbLwbYexW9uQCPo.n6uMM7dvg1o6t NrIl_R_pisx_6yvWtC_REUFp.vP2c5ex4WUzb1Ux2lZdIEcovvNCvyJ6Vjn4 bjT6U33YLvoLSdIYqI4X9lbdOqMU2Vp.EWAwh48Aju1T5IlowK4uT9vnXpLW vO.CFOYdsSUemRxPO481rJPfWIm5gylc2kJ40esll9jeeCM7sEiDzxXPi6Yc sE4blBzphrLniEHKX6sWLeHDKJ.KEKCokFDkEmAEzgaMtJ47SF_jYWqsLjHI iBtFJ9ENVhJr5ZyZ2.Ez.flcHg6uvs_VVfMul6RfIE8a5I0lnyNFEM.2Jv5c WkaffN2dxNXqL0nMTm1yPvuMofYl3T75r3qD.vuwf5UUVS7pBBWa3gyMG7EV ylXtbDGTamUSaDkRs1lnqNdEh4ZmLwgASea4uHB6xrFvTnJ08f_ifSEqzF_A ioB3F8wK.d73eWWYwEOBOJRGuxB9L6c2Y6rWDl48MaCLUwwtlaYZZW1f3_Jd _J68h973XWSLohUoS1CXFEMlVr7w- Received: from [136.187.82.86] by web171402.mail.ir2.yahoo.com via HTTP; Thu, 28 Feb 2013 02:00:06 GMT X-Rocket-MIMEInfo: 001.001,VGhhbmsgeW91IGV2ZXJ5b25lIGZvciB5b3VyIGFuc3dlci4gSXQgZ2l2ZXMgbWUgYSBsb3Qgb2YgcGF0aHMgZm9yIHJlZmxlY3Rpb24uCgpUaGFua3MgTGFycnksIEknbGwgaGF2ZSB0byBkaWcgbW9yZSBhYm91dCB0aGUgImludGVyLWNsb3VkIiBzeXN0ZW0gbXkgdW5pIGlzIHVzaW5nLgoKVGhvbWFzCgoKCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwogRGXCoDogQ2hhcmxlcyBFYXJsIDxjaGFybGVzY2VhcmxAbWUuY29tPgrDgMKgOiAidXNlckBoYWRvb3AuYXBhY2hlLm9yZyIgPHVzZXJAaGEBMAEBAQE- X-Mailer: YahooMailWebService/0.8.135.514 References: <1361969382.5766.YahooMailNeo@web171406.mail.ir2.yahoo.com> <1361974886.93875.YahooMailNeo@web171403.mail.ir2.yahoo.com> <58C11DDB-7420-4D93-B22D-D811411627D3@gmail.com> <2282C91B-9849-4DF2-80A6-C6E64A36EB66@me.com> Message-ID: <1362016806.91047.YahooMailNeo@web171402.mail.ir2.yahoo.com> Date: Thu, 28 Feb 2013 02:00:06 +0000 (GMT) From: Thomas Nguy Reply-To: Thomas Nguy Subject: Re: Work on research project "Hadoop Security Design" To: "user@hadoop.apache.org" In-Reply-To: <2282C91B-9849-4DF2-80A6-C6E64A36EB66@me.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="-293919475-397178614-1362016806=:91047" X-Virus-Checked: Checked by ClamAV on apache.org ---293919475-397178614-1362016806=:91047 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Thank you everyone for your answer. It gives me a lot of paths for reflecti= on.=0A=0AThanks Larry, I'll have to dig more about the "inter-cloud" system= my uni is using.=0A=0AThomas=0A=0A=0A=0A=0A_______________________________= _=0A De=A0: Charles Earl =0A=C0=A0: "user@hadoop.apach= e.org" =0ACc=A0: "user@hadoop.apache.org" =0AEnvoy=E9 le : Mercredi 27 f=E9vrier 2013 16h09=0AObjet= =A0: Re: Work on research project "Hadoop Security Design"=0A =0A=0AThomas,= =A0=0AInteresting distinctions articulated, thanks.=0AI knew of an effort c= alled GarewayFS that had inter-cluster secure federation as one goal, there= are some similarities to Knox.=0AC=0A=0AOn Feb 27, 2013, at 10:04 AM, Larr= y McCay wrote:=0A=0A=0AHi Thomas -=0A>=0A>=0A>I th= ink that you need articulate the problems that you want to solve for your u= niversity environment.=0A>The subject that you chose indicates "inter-cloud= environment" - so depending on the inter-cloud problems that currently exi= st for your environment there may be interesting work from the Rhino effort= or with Knox.=0A>=0A>=0A>It seems that you are leaning toward data protect= ion and encryption as a solution to some problem within your stated problem= subject.=0A>I'd be interested in the usecase that you are addressing with = it that is "inter-cloud".=0A>Another family of issues that would be interes= ting in the inter-cloud space would be various identity federation issues a= cross clouds.=0A>=0A>=0A>@Charles - by GatewayFS do you mean HttpFS and are= you asking whether Knox is related to it?=0A>If so, Knox is not directly r= elated to HttpFS though it will leverage lessons learned and hopefully the = experience of those involved.=0A>The Knox gateway is more transparent and c= ommitted to serving REST APIs to numerous Hadoop services rather than just = HDFS.=0A>The pluggable providers of Knox gateway will also facilitate easie= r integration with customer's identity infrastructure in on-prem and cloud = provider environments.=0A>=0A>=0A>Hope that helps to draw the distinction b= etween Knox and HttpFS.=0A>=0A>thanks,=0A>=0A>=0A>--larry=0A>=0A>=0A>On Wed= , Feb 27, 2013 at 9:40 AM, Charles Earl wrote:=0A= >=0A>Is this in any way related to GatewayFS?=0A>>I am also curious whether= any one knows of plans to incorporate homomorphic encryption or secure mul= tiparty into the rhino effort.=0A>>C=0A>>=0A>>=0A>>On Feb 27, 2013, at 9:30= AM, Nitin Pawar wrote:=0A>>=0A>>I am not sure if you guys have heard it or= not=A0=0A>>>=0A>>>=0A>>>HortonWorks is in process to incubate a new apache= project called Knox for hadoop security.=A0=0A>>>More on this you can look= at=A0=0A>>>=0A>>>=0A>>>http://hortonworks.com/blog/introducing-knox-hadoop= -security/=0A>>>=0A>>>=0A>>>=0A>>>http://wiki.apache.org/incubator/knox=0A>= >>=0A>>>=0A>>>=0A>>>=0A>>>On Wed, Feb 27, 2013 at 7:51 PM, Thomas Nguy wrote:=0A>>>=0A>>>Thank you very much Panshul, I'll take= a look.=0A>>>>=0A>>>>=0A>>>>Thomas.=0A>>>>=0A>>>>=0A>>>>=0A>>>>___________= _____________________=0A>>>> De=A0: Panshul Whisper = =0A>>>>=C0=A0: user@hadoop.apache.org; Thomas Nguy = =0A>>>>Envoy=E9 le : Mercredi 27 f=E9vrier 2013 13h53=0A>>>>Objet=A0: Re: W= ork on research project "Hadoop Security Design"=0A>>>> =0A>>>>=0A>>>>Hello= Thomas,=0A>>>>=0A>>>>=0A>>>>you can look into this project. This is exactl= y what you are doing, but at a larger scale.=A0=0A>>>>https://github.com/in= tel-hadoop/project-rhino/=0A>>>>=0A>>>>=0A>>>>=0A>>>>Hope this helps,=0A>>>= >=0A>>>>=0A>>>>Regards,=0A>>>>Panshul=0A>>>>=0A>>>>=0A>>>>=0A>>>>On Wed, Fe= b 27, 2013 at 1:49 PM, Thomas Nguy wrote:=0A>>>>=0A>= >>>Hello developers !=0A>>>>>=0A>>>>>=0A>>>>>I'm a student at the french un= iversity "Ensimag" and currently doing my master research on "Software secu= rity". Interested by cloud computing, I chose for subject : "Secure hadoop = cluster inter-cloud environment".=0A>>>>>My idea is to develop a framework = in order to improve the security of the Hadoop cluster running on the cloud= of my uni.=A0I have started by checking the "Hadoop research projects" pro= posed =A0on Hadoop Wiki and the following subject fits with mine:=0A>>>>>= =0A>>>>>=0A>>>>>"Hadoop Security Design:=A0=0A>>>>>An end-to-end proposal f= or how to support authentication and client side data encryption/decryption= , so that large data sets can be stored in a public HDFS and only jobs laun= ched by authenticated users can map-reduce or browse the data"=0A>>>>>=0A>>= >>>=0A>>>>>I would like to know if there are already some developers on it = so we can discuss...=A0To be honest, I'm kinda a "beginner" regarding Hadoo= p and cloud cumputing so if would be really great if you had some advices o= r hints for my research.=0A>>>>>=0A>>>>>=0A>>>>>Best regards.Thomas=A0=0A>>= >>=0A>>>>=0A>>>>=0A>>>>-- =0A>>>>=0A>>>>Regards,Ouch Whisper=0A>>>>01010101= 0101=0A>>>>=0A>>>>=0A>>>=0A>>>=0A>>>=0A>>>-- =0A>>>Nitin Pawar=0A>>>=0A>>= =0A> ---293919475-397178614-1362016806=:91047 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Thank you everyone = for your answer. It gives me a lot of paths for reflection.

Thanks Larry, I'll have to dig more about the "inter-cloud" system = my uni is using.

Thomas

<= /span>

De : Charles Earl <charlescearl@me.com>
=C0 : "user@hadoop.apache.or= g" <user@hadoop.apache.org>
Cc : "user@hadoop.apache.org" <user@hadoop.apache.org&g= t;
Envoy=E9 le : Merc= redi 27 f=E9vrier 2013 16h09
Obje= t : Re: Work on research project "Hadoop Security Design"

Thomas, 
Interest= ing distinctions articulated, thanks.
I knew of an effort called = GarewayFS that had inter-cluster secure federation as one goal, there are s= ome similarities to Knox.
C

On Feb 27, 2013, at 10:04 AM, Larry McCay = <lmccay@hortonworks.com&= gt; wrote:

Hi Thomas -

=
I think that you need articulate the problems that you want to s= olve for your university environment.
The subject that you chose = indicates "inter-cloud environment" - so depending on the inter-cloud probl= ems that currently exist for your environment there may be interesting work= from the Rhino effort or with Knox.
=0A

It seems t= hat you are leaning toward data protection and encryption as a solution to = some problem within your stated problem subject.
I'd be intereste= d in the usecase that you are addressing with it that is "inter-cloud".=0A
Another family of issues that would be interesting in the inter-c= loud space would be various identity federation issues across clouds.
=

@Charles - by GatewayFS do you mean HttpFS and are you = asking whether Knox is related to it?
=0A
If so, Knox is not direc= tly related to HttpFS though it will leverage lessons learned and hopefully= the experience of those involved.
The Knox gateway is more trans= parent and committed to serving REST APIs to numerous Hadoop services rathe= r than just HDFS.
=0A
The pluggable providers of Knox gateway will= also facilitate easier integration with customer's identity infrastructure= in on-prem and cloud provider environments.

Hope = that helps to draw the distinction between Knox and HttpFS.
=0A
thank= s,

--larry

On Wed, Feb 27, 2013 at 9:40 AM, Charles Earl <<= a rel=3D"nofollow" ymailto=3D"mailto:charles.cearl@gmail.com" target=3D"_bl= ank" href=3D"mailto:charles.cearl@gmail.com">charles.cearl@gmail.com>= ; wrote:
=0A
Is this in any way related to GatewayFS?
I a= m also curious whether any one knows of plans to incorporate homomorphic en= cryption or secure multiparty into the rhino effort.
=0A
C

On Feb 27, 2013, at 9:30 AM, Nitin Paw= ar wrote:

I am not sure= if you guys have heard it or not 
=0A
HortonWorks i= s in process to incubate a new apache project called Knox for hadoop securi= ty. 
More on this you can look at 
=0A


On Wed= , Feb 27, 2013 at 7:51 PM, Thomas Nguy <thomas.nguy@yahoo.fr> wrote:
=0A= =0A
Thank you= very much Panshul, I'll take a look.
=0A=0A

=0A=0AThomas.

=0A=0A
De := Panshul Whisper <ouchwhis= per@gmail.com>
=C0 : user@hadoop.apache.or= g;=0A Thomas Nguy <thomas.ng= uy@yahoo.fr>
Envoy=E9 le := Mercredi 27 f=E9vrier 2013 13h53
Objet : Re: Work on research project "Hadoop Sec= urity Design"
=0A=0A

Hello Thom= as,

you can look into this project. This is exactly what= you are doing, but at a larger scale. 
Hope this helps,

Regards,
P= anshul


On Wed, Feb 27, 2013 at 1:49 PM, Thomas= Nguy <thomas.= nguy@yahoo.fr> wrote:
=0A=0A=0A
<= div>=0AHello developers !

=0A
I'm a student at the french university "Ensimag" and curren= tly doing my master research on "Software security". Interested by cloud co= mputing, I chose for subject : "Secure hadoop cluster inter-cloud environme= nt".
=0A=0A=0A
My idea is to develop a framework in order to improve the security of th= e Hadoop cluster running on the cloud of my uni. I have started by checking t= he "Hadoop research projects" proposed  on Hadoop Wiki and the followi= ng subject fits with mine:
=0A=0A=0A

=0A
"Hadoop Security Design: 
=0A
=09An end-to-end proposal for how to support authentication and clien= t side data encryption/decryption, so that large data sets can be stored in= a public HDFS and only jobs launched by authenticated users can map-reduce= or browse the data"
=0A=0A=0A

=0A
I would like = to=0A know if there are already some developers on it so we can discuss...&= nbsp;To= be honest, I'm kinda a "beginner" regarding Hadoop and cloud cumputing so = if would be really great if you had some advices or hints for my research.<= /span>
=0A=0A=0A

Best regards.
=0A=0A=0A=
Thomas 
=0A



--
Regar= ds,
Ouch Whisper
010101010101
=0A
= =0A





--
Nitin Pawar
=0A
=0A


=0A

---293919475-397178614-1362016806=:91047--