hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Clay B." <...@clayb.net>
Subject Group mismatches?
Date Mon, 16 Jul 2012 19:05:55 GMT
Hi all,

I have a Hadoop cluster which uses Samba to map an Active Directory domain 
to my CentOS 5.7 Hadoop cluster. However, I notice a strange mismatch with 
groups. Does anyone have any debugging advice, or how to refresh the DFS 
groups mapping? If not, should I file a bug at 
https://issues.apache.org/jira/browse/HADOOP?

# I see the following error:
[clayb@hamster ~]$ hadoop fs -ls /projects/foobarcommander
log4j:ERROR Could not find value for key log4j.appender.NullAppender
log4j:ERROR Could not instantiate appender named "NullAppender".
ls: could not get get listing for 'hdfs://hamster:9000/projects/foobarcommander' : org.apache.hadoop.security.AccessControlException:
Permission denied: user=clayb, access=READ_EXECUTE, inode="/projects/foobarcommander":hadmin:foobarcommander:drwxrwx---

# I verify group membership -- look a mismatch!
[clayb@hamster ~]$ which groups
/usr/bin/groups
[clayb@hamster ~]$ groups
foobarcommander xxx_rec_eng domain users all all_north america batchlogon xxx-s xxx03-s xxx1-admins
xxx-emr-users xxx-emr-admins xxx1-users BUILTIN\users
[clayb@hamster ~]$ hadoop dfsgroups
log4j:ERROR Could not find value for key log4j.appender.NullAppender
log4j:ERROR Could not instantiate appender named "NullAppender".
clayb : domain users xxx_rec_eng xxx-emr-users all xxx-emr-admins batchlogon all_north america
xxx1-users xxx-s xxx03-s xxx1-admins BUILTIN\users

Notice, in particular the foobarcommander group is only shown for my 
/usr/bin/groups output. It looks like the following from the HDFS 
Permissions Guide[1] is not correct, in my case:
"The group list is the equivalent of `bash -c groups`."

# I have tried the following to no useful effect:
[admin@hamster ~]$ hadoop dfsadmin -refreshUserToGroupsMappings
log4j:ERROR Could not find value for key log4j.appender.NullAppender
log4j:ERROR Could not instantiate appender named "NullAppender".

# I do, however, see other users with the foobarcommander group, so the group should be "visible"
to Hadoop:
[clayb@hamster ~]$ hadoop dfsgroups pat
log4j:ERROR Could not find value for key log4j.appender.NullAppender
log4j:ERROR Could not instantiate appender named "NullAppender".
pat : domain users all_north america all_san diego all foobarcommander BUILTIN\users
# And 'hadoop mrgroups' (like dfsgroups) returns the same bad data, for me:
[clayb@hamster ~]$ hadoop mrgroups
log4j:ERROR Could not find value for key log4j.appender.NullAppender
log4j:ERROR Could not instantiate appender named "NullAppender".
clayb : domain users xxx_rec_eng xxx-emr-users all xxx-emr-admins batchlogon all_north america
xxx1-users xxx-s xxx03-s xxx1-admins BUILTIN\users

# And I see that the system is getting the right data via getent(1):
[clayb@hamster ~]$ getent group foobarcommander
foobarcommander:*:16777316:pat,user1,user2,user3,clayb,user4,user5,user6,user7,user8,user9,user10,user12,user13,user14,user15

# I am using Cloudera's CDH3u4 Hadoop:
[clayb@hamster ~]$ hadoop version
Hadoop 0.20.2-cdh3u4
Subversion file:///data/1/tmp/topdir/BUILD/hadoop-0.20.2-cdh3u4 -r 214dd731e3bdb687cb55988d3f47dd9e248c5690
Compiled by root on Mon May  7 14:03:02 PDT 2012
>From source with checksum a60c9795e41a3248b212344fb131c12c

I also do not see any obviously useful errors in my namenode logs.

-Clay

[1]: 
http://hadoop.apache.org/common/docs/r0.20.2/hdfs_permissions_guide.html#User+Identity


Mime
View raw message