Return-Path: X-Original-To: apmail-hadoop-mapreduce-commits-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D15D848F9 for ; Wed, 25 May 2011 21:53:12 +0000 (UTC) Received: (qmail 93768 invoked by uid 500); 25 May 2011 21:53:12 -0000 Delivered-To: apmail-hadoop-mapreduce-commits-archive@hadoop.apache.org Received: (qmail 93742 invoked by uid 500); 25 May 2011 21:53:12 -0000 Mailing-List: contact mapreduce-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-dev@hadoop.apache.org Delivered-To: mailing list mapreduce-commits@hadoop.apache.org Received: (qmail 93734 invoked by uid 99); 25 May 2011 21:53:12 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 25 May 2011 21:53:12 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 25 May 2011 21:53:05 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id BAA5123888DD; Wed, 25 May 2011 21:52:42 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1127698 - in /hadoop/mapreduce/branches/MR-279: ./ mr-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ yarn/yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/ yarn/yarn-server/yarn-server-resourcemanager/... Date: Wed, 25 May 2011 21:52:42 -0000 To: mapreduce-commits@hadoop.apache.org From: mahadev@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20110525215242.BAA5123888DD@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: mahadev Date: Wed May 25 21:52:41 2011 New Revision: 1127698 URL: http://svn.apache.org/viewvc?rev=1127698&view=rev Log: Adding job kill for any state that the job is in with access control. (mahadev) Added: hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/application/ hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/application/ApplicationACL.java hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/application/ApplicationACLsManager.java Modified: hadoop/mapreduce/branches/MR-279/CHANGES.txt hadoop/mapreduce/branches/MR-279/mr-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java hadoop/mapreduce/branches/MR-279/mr-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java hadoop/mapreduce/branches/MR-279/mr-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java hadoop/mapreduce/branches/MR-279/yarn/yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/ProtoOverHadoopRpcEngine.java hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMConfig.java hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/ApplicationMasterInfo.java hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/ApplicationsManager.java hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/ApplicationsManagerImpl.java hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/TestApplicationCleanup.java hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java Modified: hadoop/mapreduce/branches/MR-279/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/MR-279/CHANGES.txt?rev=1127698&r1=1127697&r2=1127698&view=diff ============================================================================== --- hadoop/mapreduce/branches/MR-279/CHANGES.txt (original) +++ hadoop/mapreduce/branches/MR-279/CHANGES.txt Wed May 25 21:52:41 2011 @@ -3,6 +3,7 @@ Hadoop MapReduce Change Log Trunk (unreleased changes) MAPREDUCE-279 + Adding job kill for any state that the job is in with access control. (mahadev) Added acl check for RMAdmin. (acmurthy) Modified: hadoop/mapreduce/branches/MR-279/mr-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/MR-279/mr-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java?rev=1127698&r1=1127697&r2=1127698&view=diff ============================================================================== --- hadoop/mapreduce/branches/MR-279/mr-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java (original) +++ hadoop/mapreduce/branches/MR-279/mr-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java Wed May 25 21:52:41 2011 @@ -1,20 +1,20 @@ /** -* Licensed to the Apache Software Foundation (ASF) under one -* or more contributor license agreements. See the NOTICE file -* distributed with this work for additional information -* regarding copyright ownership. The ASF licenses this file -* to you under the Apache License, Version 2.0 (the -* "License"); you may not use this file except in compliance -* with the License. You may obtain a copy of the License at -* -* http://www.apache.org/licenses/LICENSE-2.0 -* -* Unless required by applicable law or agreed to in writing, software -* distributed under the License is distributed on an "AS IS" BASIS, -* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -* See the License for the specific language governing permissions and -* limitations under the License. -*/ + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.apache.hadoop.mapred; @@ -43,6 +43,7 @@ import org.apache.hadoop.mapreduce.v2.ap import org.apache.hadoop.mapreduce.v2.api.protocolrecords.KillJobRequest; import org.apache.hadoop.mapreduce.v2.api.protocolrecords.KillTaskAttemptRequest; import org.apache.hadoop.mapreduce.v2.api.records.JobReport; +import org.apache.hadoop.mapreduce.v2.api.records.JobState; import org.apache.hadoop.mapreduce.v2.jobhistory.JHConfig; import org.apache.hadoop.net.NetUtils; import org.apache.hadoop.security.SecurityInfo; @@ -87,7 +88,7 @@ public class ClientServiceDelegate { private MRClientProtocol getProxy(ApplicationId appId, boolean forceRefresh) throws YarnRemoteException { - if (!appId.equals(currentAppId) || forceRefresh) { + if (!appId.equals(currentAppId) || forceRefresh || realProxy == null) { currentAppId = appId; refreshProxy(); } @@ -97,12 +98,15 @@ public class ClientServiceDelegate { private void refreshProxy() throws YarnRemoteException { ApplicationMaster appMaster = rm.getApplicationMaster(currentAppId); while (!ApplicationState.COMPLETED.equals(appMaster.getState()) && - !ApplicationState.FAILED.equals(appMaster.getState()) && - !ApplicationState.KILLED.equals(appMaster.getState())) { + !ApplicationState.FAILED.equals(appMaster.getState()) && + !ApplicationState.KILLED.equals(appMaster.getState()) && + !ApplicationState.ALLOCATING.equals(appMaster.getState())) { try { if (appMaster.getHost() == null || "".equals(appMaster.getHost())) { - LOG.info("AM not assigned to Job. Waiting to get the AM ..."); + LOG.debug("AM not assigned to Job. Waiting to get the AM ..."); Thread.sleep(2000); + + LOG.debug("Application state is " + appMaster.getState()); appMaster = rm.getApplicationMaster(currentAppId); continue; } @@ -113,9 +117,9 @@ public class ClientServiceDelegate { Token clientToken = new Token(); clientToken.decodeFromUrlString(clientTokenEncoded); - clientToken.setService(new Text(appMaster.getHost() + ":" - + appMaster.getRpcPort())); - UserGroupInformation.getCurrentUser().addToken(clientToken); + clientToken.setService(new Text(appMaster.getHost() + ":" + + appMaster.getRpcPort())); + UserGroupInformation.getCurrentUser().addToken(clientToken); } LOG.info("Connecting to " + serviceAddr); instantiateAMProxy(serviceAddr); @@ -126,7 +130,7 @@ public class ClientServiceDelegate { //there may be some time before AM is restarted //keep retrying by getting the address from RM LOG.info("Could not connect to " + serviceAddr + - ". Waiting for getting the latest AM address..."); + ". Waiting for getting the latest AM address..."); try { Thread.sleep(2000); } catch (InterruptedException e1) { @@ -134,12 +138,20 @@ public class ClientServiceDelegate { appMaster = rm.getApplicationMaster(currentAppId); } } - //TODO Should this be additional states ? + + /** we just want to return if its allocating, so that we dont + * block on it. This is to be able to return job status + * on a allocating Application. + */ + if (appMaster.getState() == ApplicationState.ALLOCATING) { + return; + } + if (ApplicationState.COMPLETED.equals(appMaster.getState())) { serviceAddr = conf.get(JHConfig.HS_BIND_ADDRESS, JHConfig.DEFAULT_HS_BIND_ADDRESS); LOG.info("Application state is completed. " + - "Redirecting to job history server " + serviceAddr); + "Redirecting to job history server " + serviceAddr); //TODO: serviceHttpAddr = ""; try { @@ -150,7 +162,7 @@ public class ClientServiceDelegate { } } LOG.warn("Cannot connect to Application with state " + appMaster.getState()); - throw new YarnException( + throw new YarnException( "Cannot connect to Application with state " + appMaster.getState()); } @@ -162,18 +174,18 @@ public class ClientServiceDelegate { public MRClientProtocol run() { Configuration myConf = new Configuration(conf); myConf.setClass( - CommonConfigurationKeysPublic.HADOOP_SECURITY_INFO_CLASS_NAME, - SchedulerSecurityInfo.class, SecurityInfo.class); + CommonConfigurationKeysPublic.HADOOP_SECURITY_INFO_CLASS_NAME, + SchedulerSecurityInfo.class, SecurityInfo.class); YarnRPC rpc = YarnRPC.create(myConf); return (MRClientProtocol) rpc.getProxy(MRClientProtocol.class, - NetUtils.createSocketAddr(serviceAddr), myConf); + NetUtils.createSocketAddr(serviceAddr), myConf); } }); LOG.trace("Connected to ApplicationMaster at: " + serviceAddr); } private void instantiateHistoryProxy(final String serviceAddr) - throws IOException { + throws IOException { LOG.trace("Connecting to HistoryServer at: " + serviceAddr); Configuration myConf = new Configuration(conf); //TODO This should ideally be using it's own class (instead of ClientRMSecurityInfo) @@ -186,7 +198,7 @@ public class ClientServiceDelegate { } public org.apache.hadoop.mapreduce.Counters getJobCounters(JobID arg0) throws IOException, - InterruptedException { + InterruptedException { org.apache.hadoop.mapreduce.v2.api.records.JobId jobID = TypeConverter.toYarn(arg0); try { GetCountersRequest request = recordFactory.newRecordInstance(GetCountersRequest.class); @@ -222,6 +234,13 @@ public class ClientServiceDelegate { request.setJobId(jobID); request.setFromEventId(arg1); request.setMaxEvents(arg2); + MRClientProtocol protocol = getProxy(arg0); + /** This is hack to get around the issue of faking jobstatus while the AM + * is coming up. + */ + if (protocol == null) { + return new TaskCompletionEvent[0]; + } list = getProxy(arg0).getTaskAttemptCompletionEvents(request).getCompletionEventList(); } catch(YarnRemoteException yre) {//thrown by remote server, no need to redirect LOG.warn(RPCUtil.toString(yre)); @@ -245,8 +264,8 @@ public class ClientServiceDelegate { public String[] getTaskDiagnostics(org.apache.hadoop.mapreduce.TaskAttemptID arg0) throws IOException, - InterruptedException { - + InterruptedException { + List list = null; org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID = TypeConverter.toYarn(arg0); GetDiagnosticsRequest request = recordFactory.newRecordInstance(GetDiagnosticsRequest.class); @@ -275,7 +294,7 @@ public class ClientServiceDelegate { } public JobStatus getJobStatus(JobID oldJobID) throws YarnRemoteException, - YarnRemoteException { + YarnRemoteException { org.apache.hadoop.mapreduce.v2.api.records.JobId jobId = TypeConverter.toYarn(oldJobID); LOG.debug("Getting Job status"); @@ -285,6 +304,22 @@ public class ClientServiceDelegate { GetJobReportRequest request = recordFactory.newRecordInstance(GetJobReportRequest.class); try { request.setJobId(jobId); + MRClientProtocol protocol = getProxy(oldJobID); + if (protocol == null) { + /* the protocol wasnt instantiated because the applicaton wasnt launched + * return a fake report. + */ + JobReport jobreport = recordFactory.newRecordInstance(JobReport.class); + jobreport.setCleanupProgress(0); + jobreport.setFinishTime(0); + jobreport.setJobId(jobId); + jobreport.setJobState(JobState.INITED); + jobreport.setMapProgress(0); + jobreport.setStartTime(0); + jobreport.setReduceProgress(0); + jobreport.setSetupProgress(0); + return TypeConverter.fromYarn(jobreport, jobFile, serviceHttpAddr); + } report = getProxy(oldJobID).getJobReport(request).getJobReport(); } catch(YarnRemoteException yre) {//thrown by remote server, no need to redirect LOG.warn(RPCUtil.toString(yre)); @@ -302,34 +337,34 @@ public class ClientServiceDelegate { } public org.apache.hadoop.mapreduce.TaskReport[] getTaskReports(JobID jobID, TaskType taskType) - throws YarnRemoteException, YarnRemoteException { - List taskReports = null; - org.apache.hadoop.mapreduce.v2.api.records.JobId nJobID = TypeConverter.toYarn(jobID); - GetTaskReportsRequest request = recordFactory.newRecordInstance(GetTaskReportsRequest.class); + throws YarnRemoteException, YarnRemoteException { + List taskReports = null; + org.apache.hadoop.mapreduce.v2.api.records.JobId nJobID = TypeConverter.toYarn(jobID); + GetTaskReportsRequest request = recordFactory.newRecordInstance(GetTaskReportsRequest.class); + try { + request.setJobId(nJobID); + request.setTaskType(TypeConverter.toYarn(taskType)); + taskReports = getProxy(jobID).getTaskReports(request).getTaskReportList(); + } catch(YarnRemoteException yre) {//thrown by remote server, no need to redirect + LOG.warn(RPCUtil.toString(yre)); + throw yre; + } catch(Exception e) { + LOG.debug("Failed to contact application master ", e); try { request.setJobId(nJobID); request.setTaskType(TypeConverter.toYarn(taskType)); - taskReports = getProxy(jobID).getTaskReports(request).getTaskReportList(); - } catch(YarnRemoteException yre) {//thrown by remote server, no need to redirect + taskReports = getRefreshedProxy(jobID).getTaskReports(request).getTaskReportList(); + } catch(YarnRemoteException yre) { LOG.warn(RPCUtil.toString(yre)); throw yre; - } catch(Exception e) { - LOG.debug("Failed to contact application master ", e); - try { - request.setJobId(nJobID); - request.setTaskType(TypeConverter.toYarn(taskType)); - taskReports = getRefreshedProxy(jobID).getTaskReports(request).getTaskReportList(); - } catch(YarnRemoteException yre) { - LOG.warn(RPCUtil.toString(yre)); - throw yre; - } } - return TypeConverter.fromYarn - (taskReports).toArray(new org.apache.hadoop.mapreduce.TaskReport[0]); + } + return TypeConverter.fromYarn + (taskReports).toArray(new org.apache.hadoop.mapreduce.TaskReport[0]); } public Void killJob(JobID jobID) throws YarnRemoteException, - YarnRemoteException { + YarnRemoteException { org.apache.hadoop.mapreduce.v2.api.records.JobId nJobID = TypeConverter.toYarn(jobID); KillJobRequest request = recordFactory.newRecordInstance(KillJobRequest.class); try { @@ -352,9 +387,9 @@ public class ClientServiceDelegate { } public boolean killTask(TaskAttemptID taskAttemptID, boolean fail) - throws YarnRemoteException { + throws YarnRemoteException { org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID - = TypeConverter.toYarn(taskAttemptID); + = TypeConverter.toYarn(taskAttemptID); KillTaskAttemptRequest killRequest = recordFactory.newRecordInstance(KillTaskAttemptRequest.class); FailTaskAttemptRequest failRequest = recordFactory.newRecordInstance(FailTaskAttemptRequest.class); try { Modified: hadoop/mapreduce/branches/MR-279/mr-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/MR-279/mr-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java?rev=1127698&r1=1127697&r2=1127698&view=diff ============================================================================== --- hadoop/mapreduce/branches/MR-279/mr-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java (original) +++ hadoop/mapreduce/branches/MR-279/mr-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java Wed May 25 21:52:41 2011 @@ -27,7 +27,6 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.CommonConfigurationKeys; -import org.apache.hadoop.fs.FileContext; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; import org.apache.hadoop.fs.UnsupportedFileSystemException; @@ -45,6 +44,7 @@ import org.apache.hadoop.security.Securi import org.apache.hadoop.security.UserGroupInformation; import org.apache.hadoop.security.token.Token; import org.apache.hadoop.yarn.api.ClientRMProtocol; +import org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationRequest; import org.apache.hadoop.yarn.api.protocolrecords.GetAllApplicationsRequest; import org.apache.hadoop.yarn.api.protocolrecords.GetAllApplicationsResponse; import org.apache.hadoop.yarn.api.protocolrecords.GetApplicationMasterRequest; @@ -290,6 +290,13 @@ public class ResourceMgrDelegate { LOG.info("Submitted application " + applicationId + " to ResourceManager"); return applicationId; } + + public void killApplication(ApplicationId applicationId) throws IOException { + FinishApplicationRequest request = recordFactory.newRecordInstance(FinishApplicationRequest.class); + request.setApplicationId(applicationId); + applicationsManager.finishApplication(request); + LOG.info("Killing application " + applicationId); + } public ApplicationMaster getApplicationMaster(ApplicationId appId) throws YarnRemoteException { @@ -297,18 +304,6 @@ public class ResourceMgrDelegate { request.setApplicationId(appId); GetApplicationMasterResponse response = applicationsManager.getApplicationMaster(request); ApplicationMaster appMaster = response.getApplicationMaster(); - while (appMaster.getState() != ApplicationState.RUNNING && - appMaster.getState() != ApplicationState.KILLED && - appMaster.getState() != ApplicationState.FAILED && - appMaster.getState() != ApplicationState.COMPLETED) { - appMaster = applicationsManager.getApplicationMaster(request).getApplicationMaster(); - try { - //LOG.info("Waiting for appMaster to start.."); - Thread.sleep(2000); - } catch(InterruptedException ie) { - //DO NOTHING - } - } return appMaster; } Modified: hadoop/mapreduce/branches/MR-279/mr-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/MR-279/mr-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java?rev=1127698&r1=1127697&r2=1127698&view=diff ============================================================================== --- hadoop/mapreduce/branches/MR-279/mr-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java (original) +++ hadoop/mapreduce/branches/MR-279/mr-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java Wed May 25 21:52:41 2011 @@ -51,6 +51,7 @@ import org.apache.hadoop.mapreduce.TaskC import org.apache.hadoop.mapreduce.TaskReport; import org.apache.hadoop.mapreduce.TaskTrackerInfo; import org.apache.hadoop.mapreduce.TaskType; +import org.apache.hadoop.mapreduce.TypeConverter; import org.apache.hadoop.mapreduce.filecache.DistributedCache; import org.apache.hadoop.mapreduce.protocol.ClientProtocol; import org.apache.hadoop.mapreduce.security.token.delegation.DelegationTokenIdentifier; @@ -524,7 +525,7 @@ public class YARNRunner implements Clien @Override public void killJob(JobID arg0) throws IOException, InterruptedException { - clientServiceDelegate.killJob(arg0); + resMgrDelegate.killApplication(TypeConverter.toYarn(arg0).getAppId()); } @Override Modified: hadoop/mapreduce/branches/MR-279/yarn/yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/ProtoOverHadoopRpcEngine.java URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/MR-279/yarn/yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/ProtoOverHadoopRpcEngine.java?rev=1127698&r1=1127697&r2=1127698&view=diff ============================================================================== --- hadoop/mapreduce/branches/MR-279/yarn/yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/ProtoOverHadoopRpcEngine.java (original) +++ hadoop/mapreduce/branches/MR-279/yarn/yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/ProtoOverHadoopRpcEngine.java Wed May 25 21:52:41 2011 @@ -25,7 +25,6 @@ import org.apache.hadoop.ipc.WritableRpc import org.apache.hadoop.security.UserGroupInformation; import org.apache.hadoop.security.token.SecretManager; import org.apache.hadoop.security.token.TokenIdentifier; -import org.apache.hadoop.yarn.exceptions.YarnRemoteException; import org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl; import org.apache.hadoop.yarn.ipc.RpcProtos.ProtoSpecificRpcRequest; import org.apache.hadoop.yarn.ipc.RpcProtos.ProtoSpecificRpcResponse; Modified: hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java?rev=1127698&r1=1127697&r2=1127698&view=diff ============================================================================== --- hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java (original) +++ hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java Wed May 25 21:52:41 2011 @@ -30,6 +30,7 @@ import org.apache.hadoop.conf.Configurat import org.apache.hadoop.fs.CommonConfigurationKeys; import org.apache.hadoop.net.NetUtils; import org.apache.hadoop.security.SecurityInfo; +import org.apache.hadoop.security.UserGroupInformation; import org.apache.hadoop.yarn.api.ClientRMProtocol; import org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationRequest; import org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationResponse; @@ -151,7 +152,8 @@ public class ClientRMService extends Abs public FinishApplicationResponse finishApplication(FinishApplicationRequest request) throws YarnRemoteException { ApplicationId applicationId = request.getApplicationId(); try { - applicationsManager.finishApplication(applicationId); + UserGroupInformation callerUGI = UserGroupInformation.getCurrentUser(); + applicationsManager.finishApplication(applicationId, callerUGI); } catch(IOException ie) { LOG.info("Error finishing application ", ie); } Modified: hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMConfig.java URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMConfig.java?rev=1127698&r1=1127697&r2=1127698&view=diff ============================================================================== --- hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMConfig.java (original) +++ hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMConfig.java Wed May 25 21:52:41 2011 @@ -65,4 +65,11 @@ public class RMConfig { public static final String RM_ADMIN_THREADS = YarnConfiguration.RM_PREFIX + "admin.threads"; public static final int DEFAULT_RM_ADMIN_THREADS = 1; + public static final String APPLICATION_ACL_VIEW_APP = "application.acl-view-job"; + + public static final String APPLICATION_ACL_MODIFY_APP = "application.acl-modify-job"; + + /* key for looking up the acls configuration for acls checking for application */ + public static final String RM_ACLS_ENABLED = YarnConfiguration.RM_PREFIX + + "acls.enabled"; } Modified: hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/ApplicationMasterInfo.java URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/ApplicationMasterInfo.java?rev=1127698&r1=1127697&r2=1127698&view=diff ============================================================================== --- hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/ApplicationMasterInfo.java (original) +++ hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/ApplicationMasterInfo.java Wed May 25 21:52:41 2011 @@ -98,9 +98,15 @@ public class ApplicationMasterInfo imple .addTransition(ApplicationState.PENDING, ApplicationState.FAILED, ApplicationEventType.FAILED) + .addTransition(ApplicationState.PENDING, ApplicationState.KILLED, + ApplicationEventType.KILL) + .addTransition(ApplicationState.PENDING, ApplicationState.ALLOCATING, ApplicationEventType.RECOVER, allocateTransition) - + + .addTransition(ApplicationState.PENDING, ApplicationState.ALLOCATING, + ApplicationEventType.RELEASED, new ScheduleTransition()) + .addTransition(ApplicationState.EXPIRED_PENDING, ApplicationState.ALLOCATING, ApplicationEventType.ALLOCATE, allocateTransition) @@ -110,19 +116,19 @@ public class ApplicationMasterInfo imple .addTransition(ApplicationState.EXPIRED_PENDING, ApplicationState.FAILED, ApplicationEventType.FAILED_MAX_RETRIES, failedTransition) - .addTransition(ApplicationState.PENDING, ApplicationState.CLEANUP, - ApplicationEventType.KILL, killTransition) - + .addTransition(ApplicationState.EXPIRED_PENDING, ApplicationState.KILLED, + ApplicationEventType.KILL, killTransition) + .addTransition(ApplicationState.ALLOCATING, ApplicationState.ALLOCATED, ApplicationEventType.ALLOCATED, new AllocatedTransition()) .addTransition(ApplicationState.ALLOCATING, ApplicationState.ALLOCATING, ApplicationEventType.RECOVER, allocateTransition) - .addTransition(ApplicationState.ALLOCATING, ApplicationState.CLEANUP, - ApplicationEventType.KILL, killTransition) + .addTransition(ApplicationState.ALLOCATING, ApplicationState.KILLED, + ApplicationEventType.KILL, new AllocatingKillTransition()) - .addTransition(ApplicationState.ALLOCATED, ApplicationState.CLEANUP, + .addTransition(ApplicationState.ALLOCATED, ApplicationState.KILLED, ApplicationEventType.KILL, killTransition) .addTransition(ApplicationState.ALLOCATED, ApplicationState.LAUNCHING, @@ -137,9 +143,6 @@ public class ApplicationMasterInfo imple .addTransition(ApplicationState.LAUNCHING, ApplicationState.PENDING, ApplicationEventType.LAUNCH_FAILED, failedLaunchTransition) - .addTransition(ApplicationState.PENDING, ApplicationState.ALLOCATING, - ApplicationEventType.RELEASED, new ScheduleTransition()) - /** we cant say if the application was launched or not on a recovery, so for now * we assume it was launched and wait for its restart. */ @@ -176,6 +179,9 @@ public class ApplicationMasterInfo imple .addTransition(ApplicationState.RUNNING, ApplicationState.RUNNING, ApplicationEventType.STATUSUPDATE, statusUpdatetransition) + .addTransition(ApplicationState.RUNNING, ApplicationState.KILLED, + ApplicationEventType.KILL, killTransition) + .addTransition(ApplicationState.RUNNING, ApplicationState.RUNNING, ApplicationEventType.RECOVER, new RecoverRunningTransition()) @@ -186,6 +192,9 @@ public class ApplicationMasterInfo imple ApplicationEventType.FINISH) .addTransition(ApplicationState.COMPLETED, ApplicationState.COMPLETED, + ApplicationEventType.KILL) + + .addTransition(ApplicationState.COMPLETED, ApplicationState.COMPLETED, ApplicationEventType.RECOVER) .addTransition(ApplicationState.FAILED, ApplicationState.FAILED, @@ -193,9 +202,13 @@ public class ApplicationMasterInfo imple .addTransition(ApplicationState.FAILED, ApplicationState.FAILED, ApplicationEventType.FINISH) + .addTransition(ApplicationState.FAILED, ApplicationState.FAILED, ApplicationEventType.KILL) + .addTransition(ApplicationState.KILLED, ApplicationState.KILLED, ApplicationEventType.RECOVER) - + + .addTransition(ApplicationState.KILLED, ApplicationState.KILLED, ApplicationEventType.KILL) + .addTransition(ApplicationState.KILLED, ApplicationState.KILLED, ApplicationEventType.FINISH) .installTopology(); @@ -307,6 +320,16 @@ public class ApplicationMasterInfo imple } } + private static class AllocatingKillTransition implements + SingleArcTransition> { + @Override + public void transition(ApplicationMasterInfo masterInfo, + ASMEvent event) { + masterInfo.handler.handle(new ASMEvent(ApplicationTrackerEventType.REMOVE, + masterInfo)); + } + } + private static class KillTransition implements SingleArcTransition> { @Override Modified: hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/ApplicationsManager.java URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/ApplicationsManager.java?rev=1127698&r1=1127697&r2=1127698&view=diff ============================================================================== --- hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/ApplicationsManager.java (original) +++ hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/ApplicationsManager.java Wed May 25 21:52:41 2011 @@ -23,6 +23,7 @@ import java.util.List; import org.apache.hadoop.classification.InterfaceAudience.Private; import org.apache.hadoop.classification.InterfaceStability.Evolving; +import org.apache.hadoop.security.UserGroupInformation; import org.apache.hadoop.yarn.api.records.Application; import org.apache.hadoop.yarn.api.records.ApplicationId; import org.apache.hadoop.yarn.api.records.ApplicationMaster; @@ -30,16 +31,58 @@ import org.apache.hadoop.yarn.api.record import org.apache.hadoop.yarn.server.resourcemanager.recovery.Recoverable; /** - * This interface defines the interface for ApplicationsManager. + * This interface defines the interface for ApplicationsManager. This interface + * is used by the application submission clients to call into the applications manager. */ @Private @Evolving public interface ApplicationsManager extends Recoverable { + /** + * Create and return a new application Id. + * @return a new application id + */ ApplicationId getNewApplicationID(); + + /** + * Return the {@link ApplicationMaster} information for this application. + * @param applicationId the application id of the application + * @return the {@link ApplicationMaster} for this application + */ ApplicationMaster getApplicationMaster(ApplicationId applicationId); + + /** + * Get the information for this application. + * @param applicationID the applicaiton id for the application + * @return {@link Application} information about the application. + */ Application getApplication(ApplicationId applicationID); + + /** + * Submit the application to run on the cluster. + * @param context the {@link ApplicationSubmissionContext} for this application. + * @throws IOException + */ void submitApplication(ApplicationSubmissionContext context) throws IOException; - void finishApplication(ApplicationId applicationId) throws IOException; + + /** + * Api to kill the application. + * @param applicationId the {@link ApplicationId} to be killed. + * @param callerUGI the {@link UserGroupInformation} of the user calling it. + * @throws IOException + */ + void finishApplication(ApplicationId applicationId, + UserGroupInformation callerUGI) throws IOException; + + /** + * Get all the applications in the cluster. + * This is used by the webUI. + * @return the applications in the cluster. + */ List getAllApplications(); + + /** + * Get all the applications in the cluster. + * @return the list of applications in the cluster. + */ List getApplications(); } \ No newline at end of file Modified: hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/ApplicationsManagerImpl.java URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/ApplicationsManagerImpl.java?rev=1127698&r1=1127697&r2=1127698&view=diff ============================================================================== --- hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/ApplicationsManagerImpl.java (original) +++ hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/ApplicationsManagerImpl.java Wed May 25 21:52:41 2011 @@ -19,14 +19,17 @@ package org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager; import java.io.IOException; +import java.security.AccessControlException; import java.util.ArrayList; import java.util.List; +import java.util.Map; import java.util.concurrent.atomic.AtomicInteger; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.security.UserGroupInformation; +import org.apache.hadoop.security.authorize.AccessControlList; import org.apache.hadoop.security.token.Token; import org.apache.hadoop.yarn.api.records.Application; import org.apache.hadoop.yarn.api.records.ApplicationId; @@ -36,15 +39,17 @@ import org.apache.hadoop.yarn.api.record import org.apache.hadoop.yarn.event.EventHandler; import org.apache.hadoop.yarn.factories.RecordFactory; import org.apache.hadoop.yarn.factory.providers.RecordFactoryProvider; +import org.apache.hadoop.yarn.ipc.RPCUtil; import org.apache.hadoop.yarn.security.ApplicationTokenIdentifier; import org.apache.hadoop.yarn.security.ApplicationTokenSecretManager; import org.apache.hadoop.yarn.security.client.ClientToAMSecretManager; import org.apache.hadoop.yarn.server.resourcemanager.ResourceManager; import org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.RMContext; +import org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.application.ApplicationACL; +import org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.application.ApplicationACLsManager; import org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.events.ASMEvent; import org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.events.ApplicationMasterEvents.AMLauncherEventType; import org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.events.ApplicationMasterEvents.SNEventType; -import org.apache.hadoop.yarn.server.resourcemanager.recovery.ApplicationsStore; import org.apache.hadoop.yarn.server.resourcemanager.recovery.Store.RMState; import org.apache.hadoop.yarn.server.resourcemanager.scheduler.YarnScheduler; import org.apache.hadoop.yarn.service.CompositeService; @@ -71,7 +76,8 @@ public class ApplicationsManagerImpl ext private final EventHandler eventHandler; private final ApplicationTokenSecretManager applicationTokenSecretManager; private final RMContext rmContext; - + private ApplicationACLsManager aclsManager; + private Map applicationACLs; private final RecordFactory recordFactory = RecordFactoryProvider.getRecordFactory(null); @@ -130,6 +136,8 @@ public class ApplicationsManagerImpl ext addIfService(createNewSchedulerNegotiator(scheduler)); this.amTracker = createNewAMTracker(); addIfService(amTracker); + this.aclsManager = new ApplicationACLsManager(conf); + this.applicationACLs = aclsManager.constructApplicationACLs(conf); super.init(conf); } @@ -205,11 +213,32 @@ public class ApplicationsManagerImpl ext amTracker.finish(applicationMaster.getApplicationId()); } + /** + * check if the calling user has the access to application information. + * @param applicationId + * @param callerUGI + * @param owner + * @param appACL + * @return + */ + private boolean checkAccess(UserGroupInformation callerUGI, String owner, ApplicationACL appACL) { + if (!UserGroupInformation.isSecurityEnabled()) { + return true; + } + AccessControlList applicationACL = applicationACLs.get(appACL); + return aclsManager.checkAccess(callerUGI, appACL, owner, applicationACL); + } + @Override - public synchronized void finishApplication(ApplicationId applicationId) + public synchronized void finishApplication(ApplicationId applicationId, + UserGroupInformation callerUGI) throws IOException { - /* remove the applicaiton from the scheduler for now. Later scheduler should - * be a event handler of adding and cleaning up appications*/ + ApplicationMasterInfo masterInfo = amTracker.get(applicationId); + if (!checkAccess(callerUGI, masterInfo.getUser(), ApplicationACL.MODIFY_JOB)) { + RPCUtil.getRemoteException(new AccessControlException("User " + + callerUGI.getShortUserName() + " cannot perform operation " + + ApplicationACL.MODIFY_JOB.name() + " on " + applicationId)); + } amTracker.kill(applicationId); } Added: hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/application/ApplicationACL.java URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/application/ApplicationACL.java?rev=1127698&view=auto ============================================================================== --- hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/application/ApplicationACL.java (added) +++ hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/application/ApplicationACL.java Wed May 25 21:52:41 2011 @@ -0,0 +1,56 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.application; + +import org.apache.hadoop.classification.*; +import org.apache.hadoop.yarn.server.resourcemanager.RMConfig; + +/** + * Application related ACLs + */ +@InterfaceAudience.Private +public enum ApplicationACL { + + /** + * ACL for 'viewing' application. Dictates who can 'view' some or all of the application + * related details. + */ + VIEW_JOB(RMConfig.APPLICATION_ACL_VIEW_APP), + + /** + * ACL for 'modifying' application. Dictates who can 'modify' the application for e.g., by + * killing the application + */ + MODIFY_JOB(RMConfig.APPLICATION_ACL_MODIFY_APP); + + String aclName; + + ApplicationACL(String name) { + this.aclName = name; + } + + /** + * Get the name of the ACL. Here it is same as the name of the configuration + * property for specifying the ACL for the application. + * + * @return aclName + */ + public String getAclName() { + return aclName; + } +} Added: hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/application/ApplicationACLsManager.java URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/application/ApplicationACLsManager.java?rev=1127698&view=auto ============================================================================== --- hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/application/ApplicationACLsManager.java (added) +++ hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/application/ApplicationACLsManager.java Wed May 25 21:52:41 2011 @@ -0,0 +1,104 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.application; + +import java.util.HashMap; +import java.util.Map; + +import org.apache.hadoop.classification.InterfaceAudience; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.security.AccessControlException; +import org.apache.hadoop.security.UserGroupInformation; +import org.apache.hadoop.security.authorize.AccessControlList; +import org.apache.hadoop.yarn.server.resourcemanager.RMConfig; + +@InterfaceAudience.Private +public class ApplicationACLsManager { + + Configuration conf; + + public ApplicationACLsManager(Configuration conf) { + this.conf = conf; + } + + public boolean areACLsEnabled() { + return conf.getBoolean(RMConfig.RM_ACLS_ENABLED, false); + } + + /** + * Construct the ApplicationACLs from the configuration so that they can be kept in + * the memory. If authorization is disabled on the RM, nothing is constructed + * and an empty map is returned. + * + * @return ApplicationACL to AccessControlList map. + */ + public Map constructApplicationACLs(Configuration conf) { + + Map acls = + new HashMap(); + + // Don't construct anything if authorization is disabled. + if (!areACLsEnabled()) { + return acls; + } + + for (ApplicationACL aclName : ApplicationACL.values()) { + String aclConfigName = aclName.getAclName(); + String aclConfigured = conf.get(aclConfigName); + if (aclConfigured == null) { + // If ACLs are not configured at all, we grant no access to anyone. So + // applicationOwner and superuser/supergroup _only_ can do 'stuff' + aclConfigured = " "; + } + acls.put(aclName, new AccessControlList(aclConfigured)); + } + return acls; + } + + /** + * If authorization is enabled, checks whether the user (in the callerUGI) + * is authorized to perform the operation specified by 'applicationOperation' on + * the application by checking if the user is applicationOwner or part of application ACL for the + * specific application operation. + *
    + *
  • The owner of the application can do any operation on the application
    • + *
  • For all other users/groups application-acls are checked
    • + *
+ * @param callerUGI + * @param applicationOperation + * @param applicationOwner + * @param acl + * @throws AccessControlException + */ + public boolean checkAccess(UserGroupInformation callerUGI, + ApplicationACL applicationOperation, String applicationOwner, AccessControlList acl) { + + String user = callerUGI.getShortUserName(); + if (!areACLsEnabled()) { + return true; + } + + // Allow application-owner for any operation on the application + if (user.equals(applicationOwner) + || acl.isUserAllowed(callerUGI)) { + return true; + } + + return false; + } +} Modified: hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/TestApplicationCleanup.java URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/TestApplicationCleanup.java?rev=1127698&r1=1127697&r2=1127698&view=diff ============================================================================== --- hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/TestApplicationCleanup.java (original) +++ hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/TestApplicationCleanup.java Wed May 25 21:52:41 2011 @@ -33,6 +33,7 @@ import org.apache.hadoop.conf.Configurat import org.apache.hadoop.net.NetworkTopology; import org.apache.hadoop.net.Node; import org.apache.hadoop.net.NodeBase; +import org.apache.hadoop.security.UserGroupInformation; import org.apache.hadoop.yarn.api.records.ApplicationId; import org.apache.hadoop.yarn.api.records.ApplicationState; import org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext; @@ -279,7 +280,7 @@ public class TestApplicationCleanup exte assertTrue(firstNode.getAvailableResource().getMemory() == (firstNodeMemory - (2*memoryNeeded))); ApplicationMasterInfo masterInfo = asm.getApplicationMasterInfo(appID); - asm.finishApplication(appID); + asm.finishApplication(appID, UserGroupInformation.getCurrentUser()); while (asm.launcherCleanupCalled != true) { Thread.sleep(500); } Modified: hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java?rev=1127698&r1=1127697&r2=1127698&view=diff ============================================================================== --- hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java (original) +++ hadoop/mapreduce/branches/MR-279/yarn/yarn-server/yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java Wed May 25 21:52:41 2011 @@ -29,6 +29,7 @@ import static org.apache.hadoop.test.Moc import static org.apache.hadoop.yarn.server.resourcemanager.MockNodes.*; import static org.apache.hadoop.yarn.webapp.Params.*; +import org.apache.hadoop.security.UserGroupInformation; import org.apache.hadoop.yarn.MockApps; import org.apache.hadoop.yarn.api.records.Application; import org.apache.hadoop.yarn.api.records.ApplicationId; @@ -90,7 +91,8 @@ public class TestRMWebApp { } @Override - public void finishApplication(ApplicationId applicationId) throws IOException { + public void finishApplication(ApplicationId applicationId, UserGroupInformation + callingUser) throws IOException { throw new UnsupportedOperationException("Not supported yet."); }