hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From java8964 <java8...@hotmail.com>
Subject RE: issue about run MR job use system user
Date Thu, 24 Jul 2014 14:11:53 GMT
Are you sure user 'Alex' belongs to 'hadoop' group? Why not your run command 'id alex' to prove
it? And 'Alex' belongs to 'hadoop' group can be confirmed on the namenode?
Yong

Date: Thu, 24 Jul 2014 17:11:06 +0800
Subject: issue about run MR job use system user
From: justlooks@gmail.com
To: user@hadoop.apache.org

hi,maillist:
          i create a system user on a box of my hadoop cluster ,but when i run MR job use
this user ,it get a problem, the /data directory is for mapreduce history server option, and
i also add the user into hadoop group ,since the /data privilege is 775 ,so it can write by
user in hadoop group,why still cause permssion error? anyone can help?

 

# useradd alex
 
<configuration>
        <property>
         <name>mapreduce.framework.name</name>
         <value>yarn</value>
        </property>

        <property>
         <name>mapreduce.jobhistory.address</name>
         <value>192.168.10.49:10020</value>
        </property>

        <property>
         <name>mapreduce.jobhistory.webapp.address</name>
         <value>192.168.10.49:19888</value>
        </property>

        <property>
         <name>yarn.app.mapreduce.am.staging-dir</name>
         <value>/data</value>
        </property>
.........
<configuration>

$ hadoop fs -ls /
Found 6 items
drwxrwxr-x   - hdfs  hadoop          0 2014-07-14 18:17 /data

 
$ hadoop fs -ls /data
Found 3 items
drwx------   - hdfs hadoop          0 2014-07-09 08:49 /data/hdfs
drwxrwxrwt   - hdfs hadoop          0 2014-07-08 18:52 /data/history
drwx------   - pipe hadoop          0 2014-07-14 18:17 /data/pipe

 
[alex@hz23 ~]$ id
uid=501(alex) gid=501(alex) groups=501(alex),497(hadoop)

 
 
[alex@hz23 ~]$  hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.3.0-cdh5.0.2.jar
pi 2 100
Number of Maps  = 2
Samples per Map = 100
Wrote input for Map #0
Wrote input for Map #1
Starting Job

14/07/24 17:06:23 WARN security.UserGroupInformation: PriviledgedActionException as:alex (auth:SIMPLE)
cause:org.apache.hadoop.security.AccessControlException: Permission denied: user=alex, access=WRITE,
inode="/data":hdfs:hadoop:drwxrwxr-x

        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkFsPermission(FSPermissionChecker.java:265)
        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:251)

        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:232)
        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:176)

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5490)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5472)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:5446)

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:3600)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:3570)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3544)

        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:739)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:558)

        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)

        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1986)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1982)
        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1980)

org.apache.hadoop.security.AccessControlException: Permission denied: user=alex, access=WRITE,
inode="/data":hdfs:hadoop:drwxrwxr-x
 		 	   		  
Mime
View raw message