hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-389) Infinitely assigning containers when the required resource exceeds the cluster's absolute capacity
Date Fri, 15 Feb 2013 22:45:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579562#comment-13579562
] 

Zhijie Shen commented on YARN-389:
----------------------------------

I've checked reason why the requested AM size is larger than yarn.scheduler.maximum-allocation-mb.
In fact, the AM size is not checked before requesting a container. It is because MR jobs are
submitted through ResourceMgrDelegate and managed by MRAppMaster, while checking AM size against
the maximum resource capability is only implemented in Client and ApplicationMaster of the
distributed shell.

In addition, since the requested AM size will be rounded up, it is at the risk of the roundup
value goes beyond yarn.scheduler.maximum-allocation-mb.

Anyway, the aforementioned situation is a separate issue, which I've filed in YARN-407. Here,
the focused issue is that if the requested resource is impossible to be supplied, there should
be some mechanism to stop the infinite loop of assignment. This issue seems to be similar
to YARN-56 and YARN-394.




                
> Infinitely assigning containers when the required resource exceeds the cluster's absolute
capacity
> --------------------------------------------------------------------------------------------------
>
>                 Key: YARN-389
>                 URL: https://issues.apache.org/jira/browse/YARN-389
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Zhijie Shen
>            Assignee: Zhijie Shen
>
> I've run wordcount example on branch-2 and trunk. I've set yarn.nodemanager.resource.memory-mb
to 1G and yarn.app.mapreduce.am.resource.mb to 1.5G. Therefore, resourcemanager is to assign
a 2G AM container for AM. However, the nodemanager doesn't have enough memory to assign the
container. The problem is that the assignment operation will be repeated infinitely, if the
assignment cannot be accomplished. Logs follow.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message