hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szilard Nemeth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8248) Job hangs when queue is specified and that queue has 0 capability of a resource
Date Wed, 09 May 2018 18:51:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16469291#comment-16469291
] 

Szilard Nemeth commented on YARN-8248:
--------------------------------------

Hey [~grepas]!

Both are valid points, I was thinking about the pros and cons of logging with error level
but basically I agree, from the scheduler's point of view it's not an error so I changed the
level back to info.

Also fixed the remained duplicated loggings as well.

Please check the updated patch!

Thanks!

> Job hangs when queue is specified and that queue has 0 capability of a resource
> -------------------------------------------------------------------------------
>
>                 Key: YARN-8248
>                 URL: https://issues.apache.org/jira/browse/YARN-8248
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler, yarn
>            Reporter: Szilard Nemeth
>            Assignee: Szilard Nemeth
>            Priority: Major
>         Attachments: YARN-8248-001.patch, YARN-8248-002.patch, YARN-8248-003.patch, YARN-8248-004.patch
>
>
> Job hangs when mapreduce.job.queuename is specified and the queue has 0 of any resource
(vcores / memory / other)
> In this scenario, the job should be immediately rejected upon submission since the specified
queue cannot serve the resource needs of the submitted job.
>  
> Command to run:
> {code:java}
> bin/yarn jar "./share/hadoop/mapreduce/hadoop-mapreduce-examples-$MY_HADOOP_VERSION.jar"
pi -Dmapreduce.job.queuename=sample_queue 1 1000;{code}
> fair-scheduler.xml queue config (excerpt):
>  
> {code:java}
>  <queue name="sample_queue">
>     <minResources>10000 mb,0vcores</minResources>
>     <maxResources>90000 mb,0vcores</maxResources>
>     <maxRunningApps>50</maxRunningApps>
>     <maxAMShare>-1.0f</maxAMShare>
>     <weight>2.0</weight>
>     <schedulingPolicy>fair</schedulingPolicy>
>   </queue>
> {code}
> Diagnostic message from the web UI: 
> {code:java}
> Wed May 02 06:35:57 -0700 2018] Application is added to the scheduler and is not yet
activated. (Resource request: <memory:1536, vCores:1> exceeds current queue or its parents
maximum resource allowed).{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message