hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ch huang <justlo...@gmail.com>
Subject Re: issue about run MR job use system user
Date Fri, 25 Jul 2014 02:39:15 GMT
i reslove this by make alex directory under staging directory and set the
owner to alex

On Thu, Jul 24, 2014 at 10:11 PM, java8964 <java8964@hotmail.com> wrote:

>  Are you sure user 'Alex' belongs to 'hadoop' group? Why not your run
> command 'id alex' to prove it? And 'Alex' belongs to 'hadoop' group can be
> confirmed on the namenode?
>
> Yong
>
>  ------------------------------
> Date: Thu, 24 Jul 2014 17:11:06 +0800
> Subject: issue about run MR job use system user
> From: justlooks@gmail.com
> To: user@hadoop.apache.org
>
>
> hi,maillist:
>           i create a system user on a box of my hadoop cluster ,but when i
> run MR job use this user ,it get a problem, the /data directory is for
> mapreduce history server option, and i also add the user into hadoop group
> ,since the /data privilege is 775 ,so it can write by user in hadoop
> group,why still cause permssion error? anyone can help?
>
>  # useradd alex
>
> <configuration>
>         <property>
>          <name>mapreduce.framework.name</name>
>          <value>yarn</value>
>         </property>
>         <property>
>          <name>mapreduce.jobhistory.address</name>
>          <value>192.168.10.49:10020</value>
>         </property>
>         <property>
>          <name>mapreduce.jobhistory.webapp.address</name>
>          <value>192.168.10.49:19888</value>
>         </property>
>         <property>
>          <name>yarn.app.mapreduce.am.staging-dir</name>
>          <value>/data</value>
>         </property>
> .........
> <configuration>
> $ hadoop fs -ls /
> Found 6 items
> drwxrwxr-x   - hdfs  hadoop          0 2014-07-14 18:17 /data
>
> $ hadoop fs -ls /data
> Found 3 items
> drwx------   - hdfs hadoop          0 2014-07-09 08:49 /data/hdfs
> drwxrwxrwt   - hdfs hadoop          0 2014-07-08 18:52 /data/history
> drwx------   - pipe hadoop          0 2014-07-14 18:17 /data/pipe
>
> [alex@hz23 ~]$ id
> uid=501(alex) gid=501(alex) groups=501(alex),497(hadoop)
>
>
> [alex@hz23 ~]$  hadoop jar
> /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.3.0-cdh5.0.2.jar pi 2
> 100
> Number of Maps  = 2
> Samples per Map = 100
> Wrote input for Map #0
> Wrote input for Map #1
> Starting Job
> 14/07/24 17:06:23 WARN security.UserGroupInformation:
> PriviledgedActionException as:alex (auth:SIMPLE)
> cause:org.apache.hadoop.security.AccessControlException: Permission denied:
> user=alex, access=WRITE, inode="/data":hdfs:hadoop:drwxrwxr-x
>         at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkFsPermission(FSPermissionChecker.java:265)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:251)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:232)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:176)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5490)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5472)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:5446)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:3600)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:3570)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3544)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:739)
>         at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:558)
>         at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
>         at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1986)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1982)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1980)
> org.apache.hadoop.security.AccessControlException: Permission denied:
> user=alex, access=WRITE, inode="/data":hdfs:hadoop:drwxrwxr-x
>

Mime
View raw message