hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4764) Application submission fails with NPE when submitted queue is not available in scheduler xml
Date Fri, 04 Mar 2016 17:06:41 GMT

    [ https://issues.apache.org/jira/browse/YARN-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180149#comment-15180149
] 

Sunil G commented on YARN-4764:
-------------------------------

Yes [~bibinchundatt]. Thanks for the analysis.

{code}
    if (!isRecovery && YarnConfiguration.isAclEnabled(conf)
        && scheduler instanceof CapacityScheduler &&
        !authorizer.checkPermission(new AccessRequest(
            ((CapacityScheduler) scheduler)
                .getQueue(submissionContext.getQueue()).getPrivilegedEntity(),
            userUgi, SchedulerUtils.toAccessType(QueueACL.SUBMIT_APPLICATIONS),
            submissionContext.getApplicationId().toString(),
            submissionContext.getApplicationName()))
{code}

When ACLs are enabled in cluster, as you mentioned an NPE will hit in above code since queue
is not present. This exception is now thrown out. I think this handling is not very correct
for handling non-existent queue in ACL scenario. 

Meantime in this patch, you are trying to handle this case explicitly and responding with
Exception. There are 2 cases:
1. In case queue-mappings, we can submit w/o queue. So I think we might break some feature
here now? Thoughts?
2. Do we need to send this as RPC remote exception from here?

Pls correct me if I am wrong.


> Application submission fails with NPE when submitted queue is not available in scheduler
xml
> --------------------------------------------------------------------------------------------
>
>                 Key: YARN-4764
>                 URL: https://issues.apache.org/jira/browse/YARN-4764
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Bibin A Chundatt
>            Assignee: Bibin A Chundatt
>         Attachments: 0001-YARN-4764.patch
>
>
> Available queues in capacity scheduler 
> -root
> --queue1
> --queue2
> Submit application with queue3
> {noformat}
> 16/03/04 16:40:08 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1457077554812_1901
> 16/03/04 16:40:08 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN, Service:
ha-hdfs:hacluster, Ident: (HDFS_DELEGATION_TOKEN token 3938 for mapred with renewer yarn)
> 16/03/04 16:40:08 WARN retry.RetryInvocationHandler: Exception while invoking class org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.submitApplication
over rm2. Not retrying because try once and fail.
> java.lang.NullPointerException: java.lang.NullPointerException
>         at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:366)
>         at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.submitApplication(RMAppManager.java:289)
>         at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitApplication(ClientRMService.java:618)
>         at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitApplication(ApplicationClientProtocolPBServiceImpl.java:252)
>         at org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:483)
>         at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2305)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2301)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:422)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2301)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
>         at org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
>         at org.apache.hadoop.yarn.ipc.RPCUtil.instantiateRuntimeException(RPCUtil.java:85)
>         at org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:122)
>         at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.submitApplication(ApplicationClientProtocolPBClientImpl.java:272)
> {noformat}
> Should be queue doesnt exist



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message