hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amir Mal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2435) Capacity scheduler should only allow Kill Application Requests from ADMINISTER_QUEUE users
Date Thu, 21 Aug 2014 12:40:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14105337#comment-14105337
] 

Amir Mal commented on YARN-2435:
--------------------------------

from the resourcemanager log file:
{quote}
2014-08-21 14:37:21,550 INFO org.apache.hadoop.yarn.server.resourcemanager.ClientRMService:
Allocated new applicationId: 4
2014-08-21 14:37:22,573 INFO org.apache.hadoop.yarn.server.resourcemanager.ClientRMService:
Application with id 4 submitted by user user1
2014-08-21 14:37:22,573 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger:
USER=user1    IP=9.148.29.243 OPERATION=Submit Application Request    TARGET=ClientRMService
 RESULT=SUCCESS  APPID=application_1408540602935_0004                                    
                                                                                         
                                                 
2014-08-21 14:37:22,573 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
Storing application with id application_1408540602935_0004                               
                        
2014-08-21 14:37:22,573 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
application_1408540602935_0004 State change from NEW to NEW_SAVING                       
                        
2014-08-21 14:37:22,573 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore:
Storing info for app: application_1408540602935_0004                                     
                  
2014-08-21 14:37:22,573 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
application_1408540602935_0004 State change from NEW_SAVING to SUBMITTED                 
                        
2014-08-21 14:37:22,578 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue:
Application added - appId: application_1408540602935_0004 user: user1 leaf-queue of parent:
root #applications: 1                                                                    
                                                                                         
                                 
2014-08-21 14:37:22,578 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler:
Accepted application application_1408540602935_0004 from {color:red}user: user1, in queue:
default{color}      
2014-08-21 14:37:22,579 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
application_1408540602935_0004 State change from SUBMITTED to ACCEPTED                   
                        
2014-08-21 14:37:22,579 INFO org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService:
Registering app attempt : appattempt_1408540602935_0004_000001                           
               
2014-08-21 14:37:22,579 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
appattempt_1408540602935_0004_000001 State change from NEW to SUBMITTED                  
         
2014-08-21 14:37:22,580 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue:
Application application_1408540602935_0004 from user: user1 activated in queue: default  
           
2014-08-21 14:37:22,581 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue:
Application added - appId: application_1408540602935_0004 user: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User@2690c123,
leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications:
0 #queue-active-applications: 1              
2014-08-21 14:37:22,581 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler:
Added Application Attempt appattempt_1408540602935_0004_000001 to scheduler from user user1
in queue default                                                                         
                                                                                         
                           
2014-08-21 14:37:22,582 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
appattempt_1408540602935_0004_000001 State change from SUBMITTED to SCHEDULED            
         
2014-08-21 14:37:23,209 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl:
container_1408540602935_0004_01_000001 Container Transitioned from NEW to ALLOCATED      
            
2014-08-21 14:37:23,210 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger:
USER=user1    OPERATION=AM Allocated Container        TARGET=SchedulerApp     RESULT=SUCCESS
 APPID=application_1408540602935_0004  CONTAINERID=container_1408540602935_0004_01_000001
...
2014-08-21 14:37:54,564 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger:
USER={color:red}user2{color}    IP=9.148.29.244 OPERATION=Kill Application Request      TARGET=ClientRMService
 RESULT=SUCCESS  APPID=application_1408540602935_0004
{quote}

> Capacity scheduler should only allow Kill Application Requests from ADMINISTER_QUEUE
users
> ------------------------------------------------------------------------------------------
>
>                 Key: YARN-2435
>                 URL: https://issues.apache.org/jira/browse/YARN-2435
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler
>    Affects Versions: 2.5.0, 2.4.1
>         Environment: Red Hat Enterprise Linux Server release 6.4 (Santiago);  Linux 2.6.32-358.el6.x86_64
GNU/Linux; 
> $JAVA_HOME/bin/java -version
> java version "1.7.0_55"
> OpenJDK Runtime Environment (rhel-2.4.7.1.el6_5-x86_64 u55-b13)
> OpenJDK 64-Bit Server VM (build 24.51-b03, mixed mode)
>            Reporter: Amir Mal
>
> A user without ADMINISTER_QUEUE privilege can kill application from all queues.
> to replicate the bug:
> 1) install cluster with {{yarn.resourcemanager.scheduler.class}} set to org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.*CapacityScheduler*
> 2) created 2 users (user1, user2) each belong to a separate group (group1, group2)
> 3) set {{acl_submit_applications}} and {{acl_administer_queue}} of the {{root}} and {{root.default}}
queues to group1
> 4) submit job to {{default}} queue by user1
> {quote}
> [user1@htc2n3 ~]$ mapred  queue -showacls
> ...
> Queue acls for user :  user1
> Queue  Operations
> =====================
> root  ADMINISTER_QUEUE,SUBMIT_APPLICATIONS
> default  ADMINISTER_QUEUE,SUBMIT_APPLICATIONS
> [user1@htc2n3 ~]$ yarn  jar /opt/apache/hadoop-2.5.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.4.1.jar
pi -Dmapreduce.job.queuename=default 4 1000000000
> {quote}
> 5) kill the application by user2
> {quote}
> [user2@htc2n4 ~]$ mapred  queue -showacls
> ...
> Queue acls for user :  user2
> Queue  Operations
> =====================
> root
> default
> [user2@htc2n4 ~]$ yarn application -kill application_1408540602935_0004
> ...
> Killing application application_1408540602935_0004
> 14/08/21 14:37:54 INFO impl.YarnClientImpl: Killed application application_1408540602935_0004
> {quote}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message