hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zheng, Kai" <kai.zh...@intel.com>
Subject RE: Secure hadoop and group permission on HDFS
Date Tue, 16 Oct 2012 03:07:34 GMT
Hi Koert & Harsh,

Regarding LdapGroupsMapping, I have questions:

1.      Is it possible to use ShellBasedUnixGroupsMapping for Hadoop service principals/users,
and LdapGroupsMapping for end user accounts?

In our  environment, normal end users (along with their groups info) for Hadoop cluster are
from AD, and for them we should use the ldap mapping;

but for hdfs/mapred service principals, the default shell based one is enough, and we don't
want to create the user/group entries in AD just for that.

2.      Can we support multiple ADs? Hadoop users might come from more than ONE AD in big

3.      Is there any technical  issue not to support LDAPs like OpenLDAP? In my understanding,
one possible difficulity might be that it's not easy to extract common

group lookup mechanism with common filters/configurations both for AD and OpenLDAP like, right?

I'm wondering if these are just limits for current implementation, and if so if we need to
improve that. Might the community has already been going for that?



From: Ivan Frain [mailto:ivan.frain@gmail.com]
Sent: Tuesday, October 09, 2012 2:46 PM
To: user@hadoop.apache.org
Subject: Re: Secure hadoop and group permission on HDFS

Hi Koert,

Another option is to use the LdapGroupsMapping which picks up the group membership from a
LDAP directory.
You can find more details on the JIRA issue: https://issues.apache.org/jira/browse/HADOOP-8121
Up to now, it is available for ActiveDirectory and released in hadoop-2.0.0-alpha and next
You can easily apply the patch on a 0.23.1, I already did that and it works well.

OpenLdap with POSIX groups is not yet supported by this patch, it was tailored for ActiveDirectory.


2012/10/9 Harsh J <harsh@cloudera.com<mailto:harsh@cloudera.com>>

If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
class (via hadoop.security.group.mapping), then yes the NameNode's
view of the local unix groups (and the primary group) of the user is
the final say on what groups the user belongs to. This can be relied
on - but note that HDFS uses BSD style semantics when it comes to
groups and when creating directories/files, the parent directory
groups are inherited automatically unless altered after creation.

On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <koert@tresata.com<mailto:koert@tresata.com>>
> With secure hadoop the user name is authenticated by the kerberos server.
> But what about the groups that the user is a member of? Are these simple the
> groups that the user is a member of on the namenode machine?
> Is it viable to manage access to files on HDFS using groups on a secure
> hadoop cluster?

Harsh J

Ivan Frain
11, route de Grenade
31530 Saint-Paul-sur-Save
mobile: +33 (0)6 52 52 47 07

View raw message