hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (YARN-6543) yarn application's privilege is determined by yarn process creator instead of yarn application user.
Date Wed, 12 Jul 2017 21:46:00 GMT

     [ https://issues.apache.org/jira/browse/YARN-6543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vinod Kumar Vavilapalli resolved YARN-6543.
-------------------------------------------
    Resolution: Not A Bug

Agree with [~rohithsharma], this is working as designed. Closing this.

> yarn application's privilege is determined by yarn process creator instead of yarn application
user.
> ----------------------------------------------------------------------------------------------------
>
>                 Key: YARN-6543
>                 URL: https://issues.apache.org/jira/browse/YARN-6543
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: wuchang
>
> My application is a pyspark application which is impersonated by user 'wuchang'
> My application infomation is :
> {code}
> Application Report : 
>         Application-Id : application_1493004858240_0007
>         Application-Name : livy-session-6
>         Application-Type : SPARK
>         User : wuchang
>         Queue : root.wuchang
>         Start-Time : 1493708942748
>         Finish-Time : 0
>         Progress : 10%
>         State : RUNNING
>         Final-State : UNDEFINED
>         Tracking-URL : http://10.120.241.82:34462
>         RPC Port : 0
>         AM Host : 10.120.241.82
>         Aggregate Resource Allocation : 4369480 MB-seconds, 2131 vcore-seconds
>         Diagnostics :
> {code}
> And the process is :
> {code}
> appuser  25454 25872  0 15:09 ?        00:00:00 bash /data/data/hadoop/tmp/nm-local-dir/usercache/wuchang/appcache/application_1493004858240_0007/container_1493004858240_0007_01_000004/default_container_executor.sh
> appuser  25456 25454  0 15:09 ?        00:00:00 /bin/bash -c /home/jdk/bin/java -server
-Xmx1024m -Djava.io.tmpdir=/data/data/hadoop/tmp/nm-local-dir/usercache/wuchang/appcache/application_1493004858240_0007/container_1493004858240_0007_01_000004/tmp
'-Dspark.ui.port=0' '-Dspark.driver.port=40969' -Dspark.yarn.app.container.log.dir=/home/log/hadoop/logs/userlogs/application_1493004858240_0007/container_1493004858240_0007_01_000004
-XX:OnOutOfMemoryError='kill %p' org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url
spark://CoarseGrainedScheduler@10.120.241.82:40969 --executor-id 2 --hostname 10.120.241.18
--cores 1 --app-id application_1493004858240_0007 --user-class-path file:/data/data/hadoop/tmp/nm-local-dir/usercache/wuchang/appcache/application_1493004858240_0007/container_1493004858240_0007_01_000004/__app__.jar
--user-class-path file:/data/data/hadoop/tmp/nm-local-dir/usercache/wuchang/appcache/application_1493004858240_0007/container_1493004858240_0007_01_000004/livy-api-0.3.0-SNAPSHOT.jar
--user-class-path file:/data/data/hadoop/tmp/nm-local-dir/usercache/wuchang/appcache/application_1493004858240_0007/container_1493004858240_0007_01_000004/livy-rsc-0.3.0-SNAPSHOT.jar
--user-class-path file:/data/data/hadoop/tmp/nm-local-dir/usercache/wuchang/appcache/application_1493004858240_0007/container_1493004858240_0007_01_000004/netty-all-4.0.29.Final.jar
--user-class-path file:/data/data/hadoop/tmp/nm-local-dir/usercache/wuchang/appcache/application_1493004858240_0007/container_1493004858240_0007_01_000004/commons-codec-1.9.jar
--user-class-path file:/data/data/hadoop/tmp/nm-local-dir/usercache/wuchang/appcache/application_1493004858240_0007/container_1493004858240_0007_01_000004/livy-core_2.11-0.3.0-SNAPSHOT.jar
--user-class-path file:/data/data/hadoop/tmp/nm-local-dir/usercache/wuchang/appcache/application_1493004858240_0007/container_1493004858240_0007_01_000004/livy-repl_2.11-0.3.0-SNAPSHOT.jar
1> /home/log/hadoop/logs/userlogs/application_1493004858240_0007/container_1493004858240_0007_01_000004/stdout
2> /home/log/hadoop/logs/userlogs/application_1493004858240_0007/container_1493004858240_0007_01_000004/stderr
> appuser  25468 25456  2 15:09 ?        00:00:09 /home/jdk/bin/java -server -Xmx1024m
-Djava.io.tmpdir=/data/data/hadoop/tmp/nm-local-dir/usercache/wuchang/appcache/application_1493004858240_0007/container_1493004858240_0007_01_000004/tmp
-Dspark.ui.port=0 -Dspark.driver.port=40969 -Dspark.yarn.app.container.log.dir=/home/log/hadoop/logs/userlogs/application_1493004858240_0007/container_1493004858240_0007_01_000004
-XX:OnOutOfMemoryError=kill %p org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url
spark://CoarseGrainedScheduler@10.120.241.82:40969 --executor-id 2 --hostname 10.120.241.18
--cores 1 --app-id application_1493004858240_0007 --user-class-path file:/data/data/hadoop/tmp/nm-local-dir/usercache/wuchang/appcache/application_1493004858240_0007/container_1493004858240_0007_01_000004/__app__.jar
--user-class-path file:/data/data/hadoop/tmp/nm-local-dir/usercache/wuchang/appcache/application_1493004858240_0007/container_1493004858240_0007_01_000004/livy-api-0.3.0-SNAPSHOT.jar
--user-class-path file:/data/data/hadoop/tmp/nm-local-dir/usercache/wuchang/appcache/application_1493004858240_0007/container_1493004858240_0007_01_000004/livy-rsc-0.3.0-SNAPSHOT.jar
--user-class-path file:/data/data/hadoop/tmp/nm-local-dir/usercache/wuchang/appcache/application_1493004858240_0007/container_1493004858240_0007_01_000004/netty-all-4.0.29.Final.jar
--user-class-path file:/data/data/hadoop/tmp/nm-local-dir/usercache/wuchang/appcache/application_1493004858240_0007/container_1493004858240_0007_01_000004/commons-codec-1.9.jar
--user-class-path file:/data/data/hadoop/tmp/nm-local-dir/usercache/wuchang/appcache/application_1493004858240_0007/container_1493004858240_0007_01_000004/livy-core_2.11-0.3.0-SNAPSHOT.jar
--user-class-path file:/data/data/hadoop/tmp/nm-local-dir/usercache/wuchang/appcache/application_1493004858240_0007/container_1493004858240_0007_01_000004/livy-repl_2.11-0.3.0-SNAPSHOT.jar
> appuser  26936 25846  0 15:16 pts/0    00:00:00 grep --color=auto application_1493004858240_0007
> {code}
> The main problem is that the application user is "wuchang" , but the yarn application
is created by my OS super-user "appuser" , so , the privilege becomes the problem. My code
always run as the privilege of appuser instead of "wuchang".
> For example , below is the pyspark code:
> {code}
> import os
> os.system("hadoop fs -rm -r /user/appuser/test.dat")
> {code}
> user "wuchang" should not have privilege to remove the file test.dat which located in
the home directory of appuser. But since the yarn application process is created by "appuser",
it does, although the yarn application user is "wuchang".



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org


Mime
View raw message