hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-389) Infinitely assigning containers when the required resource exceeds the cluster's absolute capacity
Date Wed, 24 Jul 2013 16:57:56 GMT

    [ https://issues.apache.org/jira/browse/YARN-389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13718571#comment-13718571
] 

Bikas Saha commented on YARN-389:
---------------------------------

We should check for container level requests during allocate RPC request processing. The check
that container resource > resource in the cluster is probably less useful compared to checking
the container resource > resource on the largest machine.

I think its fine to be able to reject requests later on if they cannot be fulfilled because
the cluster has changed state. eg. the only machine with enough resources for a container
has failed or container wants a specific machine and that has failed. Thats why I had linked
this jira to YARN-394.
                
> Infinitely assigning containers when the required resource exceeds the cluster's absolute
capacity
> --------------------------------------------------------------------------------------------------
>
>                 Key: YARN-389
>                 URL: https://issues.apache.org/jira/browse/YARN-389
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Zhijie Shen
>            Assignee: Omkar Vinit Joshi
>
> I've run wordcount example on branch-2 and trunk. I've set yarn.nodemanager.resource.memory-mb
to 1G and yarn.app.mapreduce.am.resource.mb to 1.5G. Therefore, resourcemanager is to assign
a 2G AM container for AM. However, the nodemanager doesn't have enough memory to assign the
container. The problem is that the assignment operation will be repeated infinitely, if the
assignment cannot be accomplished. Logs follow.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message