Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A206D17B79 for ; Wed, 3 Jun 2015 12:04:05 +0000 (UTC) Received: (qmail 97258 invoked by uid 500); 3 Jun 2015 12:04:00 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 97139 invoked by uid 500); 3 Jun 2015 12:04:00 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 97129 invoked by uid 99); 3 Jun 2015 12:04:00 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Jun 2015 12:04:00 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id F36E81A4400 for ; Wed, 3 Jun 2015 12:03:59 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3 X-Spam-Level: *** X-Spam-Status: No, score=3 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=3, SPF_HELO_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id O_cO_YQmBYE0 for ; Wed, 3 Jun 2015 12:03:51 +0000 (UTC) Received: from emea01-am1-obe.outbound.protection.outlook.com (mail-am1on0103.outbound.protection.outlook.com [157.56.112.103]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id D80B642AD6 for ; Wed, 3 Jun 2015 12:03:50 +0000 (UTC) Received: from AM3PR04MB1474.eurprd04.prod.outlook.com (25.163.186.20) by AM3PR04MB1476.eurprd04.prod.outlook.com (25.163.186.22) with Microsoft SMTP Server (TLS) id 15.1.172.22; Wed, 3 Jun 2015 12:03:07 +0000 Received: from AM3PR04MB1474.eurprd04.prod.outlook.com ([25.163.186.20]) by AM3PR04MB1474.eurprd04.prod.outlook.com ([25.163.186.20]) with mapi id 15.01.0184.014; Wed, 3 Jun 2015 12:03:06 +0000 From: Nathaniel Braun To: "user@hadoop.apache.org" Subject: RE: HTTPFS without impersonation Thread-Topic: HTTPFS without impersonation Thread-Index: AdCd4A9JjcL8buQ2RQaDpAlhIAbI1wAEhYuAAAAWhEAAAHp5AAAADmTA Date: Wed, 3 Jun 2015 12:03:06 +0000 Message-ID: References: <4763CFFD-015D-4BBF-AD48-A8E6D71794BF@hortonworks.com> In-Reply-To: <4763CFFD-015D-4BBF-AD48-A8E6D71794BF@hortonworks.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=n.braun@criteo.com; x-originating-ip: [91.199.242.236] x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:AM3PR04MB1476; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(601004)(520003)(5005006)(3002001);SRVR:AM3PR04MB1476;BCL:0;PCL:0;RULEID:;SRVR:AM3PR04MB1476; x-forefront-prvs: 05961EBAFC x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(189002)(24454002)(53754006)(377454003)(164054003)(40764003)(199003)(93886004)(64706001)(97736004)(102836002)(15975445007)(33656002)(76576001)(4001540100001)(2900100001)(68736005)(105586002)(50986999)(19300405004)(2501003)(5002640100001)(66066001)(86362001)(76176999)(54356999)(2950100001)(62966003)(46102003)(19580395003)(81156007)(19625215002)(92566002)(77156002)(450100001)(106356001)(2351001)(74316001)(19580405001)(189998001)(19609705001)(16236675004)(107886002)(40100003)(5001830100001)(5001960100002)(87936001)(5001860100001)(101416001)(122556002)(2656002)(110136002);DIR:OUT;SFP:1102;SCL:1;SRVR:AM3PR04MB1476;H:AM3PR04MB1474.eurprd04.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; received-spf: None (protection.outlook.com: criteo.com does not designate permitted sender hosts) Content-Type: multipart/alternative; boundary="_000_AM3PR04MB1474E49231A8D616484E08E299B40AM3PR04MB1474eurp_" MIME-Version: 1.0 X-OriginatorOrg: criteo.com X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Jun 2015 12:03:06.6045 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 2a35d8fd-574d-48e3-927c-8c398e225a01 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM3PR04MB1476 --_000_AM3PR04MB1474E49231A8D616484E08E299B40AM3PR04MB1474eurp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, We want to let users & teams be able to run their HTTPFS in order to isolat= e instances. One team thus cannot crash another team's HTTPFS instance. Now, I make the following request: curl "localhost:14000/webhdfs/v1/user/team_user?op=3DLISTSTATUS&user.name= =3Dteam_user" And I get the following response: {"RemoteException":{"message":"User: team_user is not allowed to impersonat= e team_user","exception":"RemoteException","javaClassName":"org.apache.hado= op.ipc.RemoteException"}} Thanks, Nathaniel From: Larry McCay [mailto:lmccay@hortonworks.com] Sent: mercredi 3 juin 2015 13:57 To: user@hadoop.apache.org Subject: Re: HTTPFS without impersonation Out of curiosity, what is the added benefit of having HttpFs run as separat= e team users give you? If the APIs are invoked with SPNEGO or a user.name of the appropriate user = don't you get the same permissions based protections? Generally speaking, gateways such as HttpFs provide access on behalf of end= users. On Jun 3, 2015, at 7:44 AM, Nathaniel Braun > wrote: Hi, Thanks for your answer. With this setup, only the HTTP user will be able to impersonate other users= , so HTTPFS has to run with the HTTP user. Instead, I need users to run HTTPFS with their own user, not with the HTTP = user. Thanks From: Wellington Chevreuil [mailto:wellington.chevreuil@gmail.com] Sent: mercredi 3 juin 2015 13:41 To: user@hadoop.apache.org Subject: Re: HTTPFS without impersonation Hi, do u have below property on core-site.xml file used by your hdfs? hadoop.proxyuser.HTTP.hosts * hadoop.proxyuser.HTTP.groups * Hello all, We need to run several HTTPFS instances on our Hadoop cluster, with differe= nt users (basically, one HTTPFS per team). In our setup, each HTTPFS instance runs as a team user and is allowed write= access to that user's directory only (so, HTTPFS does not run as the httpf= s user). However, this setup does not work, as we get exceptions related to imperson= ation, such as this one: {"RemoteException":{"message":"User: team_user is not allowed to impersonat= eteam_user","exception":"RemoteException","javaClassName":"org.apache.hadoo= p.ipc.RemoteException"}} So, it seems that HTTPFS unconditionally tries to impersonate a user, even = though it's running as that same user. Is there a way to somehow disable im= personation? Thanks for your help. Regards, Nathaniel --_000_AM3PR04MB1474E49231A8D616484E08E299B40AM3PR04MB1474eurp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi,

 

We want to let users & teams be a= ble to run their HTTPFS in order to isolate instances. One team thus cannot= crash another team’s HTTPFS instance.

 

Now, I make the following request:

 

curl "localhost:14000/webhdfs/v1/user/team_user= ?op=3DLISTSTATUS&user.name=3Dteam_user"

 

And I get the following response:

 

{"RemoteException":{"message":"Use= r: team_user is not allowed to impersonate team_user","= ;exception":"RemoteException","javaClassName":&quo= t;org.apache.hadoop.ipc.RemoteException"}}

 

Thanks,

Nathaniel

 

From: Larry McCay [mailto:lmccay@hor= tonworks.com]
Sent: mercredi 3 juin 2015 13:57
To: user@hadoop.apache.org
Subject: Re: HTTPFS without impersonation

 

Out of curiosity, what is the added benefit of havin= g HttpFs run as separate team users give you?

If the APIs are invoked with SPNEGO or a user.name o= f the appropriate user don’t you get the same permissions based prote= ctions?

 

Generally speaking, gateways such as HttpFs provide = access on behalf of endusers.

 

On Jun 3, 2015, at 7:44 AM, Nathaniel Braun <n.braun@criteo.com> wrote:



Hi,

 

Thanks for your answer.

 

With this setup, only the HTTP user will be able to impersonate other users, so HTTPFS has to run with the=  HTTP = user.

 

Instead, I need users to run HTTPFS w= ith their own user, not with the HTTP user.

 

Thanks

 

From: Wellington Chevreuil [mailto:wellin= gton.chevreuil@gmail.com] 
Sent: mercredi 3 j= uin 2015 13:41
To: user@hadoop.apache.org
Subject: Re: HTTPF= S without impersonation

 

Hi, do u have below property on core-site.xml file used by your hdfs?

<property>
    <name>hadoop.proxyuser.HTTP.hosts</name>
    <value>*</value>
  </property>
  <property>
    <name>hadoop.proxyuser.HTTP.groups</name>     <value>*</value>
  </property>

Hello all,

 

We need to run several HTTPFS instances on our Hadoo= p cluster, with different users (basically, one HTTPFS per team).

 

In our setup, each HTTPFS instance runs as a team us= er and is allowed write access to that user’s directory only (so, HTT= PFS does not run as the httpfs user).

 

However, this setup does not work, as we get excepti= ons related to impersonation, such as this one:

 

{"RemoteEx= ception":{"message":"User: team_user is not allowed to impersonateteam_user","exc= eption":"RemoteException","javaClassName":"or= g.apache.hadoop.ipc.RemoteException"}}

 

So, it seems that HTTPFS unconditionally tries to= impersonate a user, even though it’s running as that same user. = Is there a way to somehow disable impersonation?

 

Thanks for your help.

 

Regards,

Nathaniel

 

--_000_AM3PR04MB1474E49231A8D616484E08E299B40AM3PR04MB1474eurp_--