hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-389) Infinitely assigning containers when the required resource exceeds the cluster's absolute capacity
Date Wed, 24 Jul 2013 18:25:50 GMT

    [ https://issues.apache.org/jira/browse/YARN-389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13718666#comment-13718666

Bikas Saha commented on YARN-389:

I dont think we can be good at predicting the future and so it may not be useful to add more
logic that allows unsatisfiable requests for some time before rejecting them. We should reject
requests when we see that they cannot be allocated. The current example that works in the
code is when the request is for resources > than configured maximum. Later on, when requests
become unsatisfiable then we need a way reject requests and inform the AM during the allocate
heartbeat. This is tracked by YARN-394.

For this jira itself, is it correct to say that the cluster was misconfigured? MR AM container
request was set to 1.5G which is greater than memory on NM 1G. What was the value of YarnConfiguration.RM_SCHEDULER_MAXIMUM_ALLOCATION_MB?
That should have been <= 1G. Then the request should have been rejected by the RM ApplicationMasterService.
> Infinitely assigning containers when the required resource exceeds the cluster's absolute
> --------------------------------------------------------------------------------------------------
>                 Key: YARN-389
>                 URL: https://issues.apache.org/jira/browse/YARN-389
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Zhijie Shen
>            Assignee: Omkar Vinit Joshi
> I've run wordcount example on branch-2 and trunk. I've set yarn.nodemanager.resource.memory-mb
to 1G and yarn.app.mapreduce.am.resource.mb to 1.5G. Therefore, resourcemanager is to assign
a 2G AM container for AM. However, the nodemanager doesn't have enough memory to assign the
container. The problem is that the assignment operation will be repeated infinitely, if the
assignment cannot be accomplished. Logs follow.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message