hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-389) Infinitely assigning containers when the required resource exceeds the cluster's absolute capacity
Date Fri, 22 Mar 2013 21:17:15 GMT

    [ https://issues.apache.org/jira/browse/YARN-389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13611266#comment-13611266
] 

Zhijie Shen commented on YARN-389:
----------------------------------

@Bikas, the problem was produced by two issues:

1. The problematic configuration, which made the AM container require more resource than a
node had (In my previous comments, I wrongly mentioned the entire cluster resource, because
I was testing it on one-node cluster, such that the maximum cluster resource is equal to that
of a single node). This issue can be eliminated when YARN-193 gets fixed.

2. Anyway, when the request is impossible to be fulfilled (e.g., the requested resource is
more than any node has), it should be somehow prohibited instead of being executed again and
again. YARN-56 and YARN-394 are the two tickets that want to solve the similar issue.
                
> Infinitely assigning containers when the required resource exceeds the cluster's absolute
capacity
> --------------------------------------------------------------------------------------------------
>
>                 Key: YARN-389
>                 URL: https://issues.apache.org/jira/browse/YARN-389
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Zhijie Shen
>            Assignee: Zhijie Shen
>
> I've run wordcount example on branch-2 and trunk. I've set yarn.nodemanager.resource.memory-mb
to 1G and yarn.app.mapreduce.am.resource.mb to 1.5G. Therefore, resourcemanager is to assign
a 2G AM container for AM. However, the nodemanager doesn't have enough memory to assign the
container. The problem is that the assignment operation will be repeated infinitely, if the
assignment cannot be accomplished. Logs follow.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message