hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Giovanni Matteo Fumarola (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-110) AM releases too many containers due to the protocol
Date Mon, 21 Dec 2015 19:30:46 GMT

    [ https://issues.apache.org/jira/browse/YARN-110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15066946#comment-15066946
] 

Giovanni Matteo Fumarola commented on YARN-110:
-----------------------------------------------

[~kasha] My idea is to keep a list of requested containers in AppSchedulingInfo. 
When the RM sends containers to the AM and in the same heartbeat the AM asks containers, the
adding check forwards to the capacity scheduler the correct number of containers.

After the vacation I will rebase my patch and I will push it.

> AM releases too many containers due to the protocol
> ---------------------------------------------------
>
>                 Key: YARN-110
>                 URL: https://issues.apache.org/jira/browse/YARN-110
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager, scheduler
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>         Attachments: YARN-110.patch
>
>
> - AM sends request asking 4 containers on host H1.
> - Asynchronously, host H1 reaches RM and gets assigned 4 containers. RM at this point,
sets the value against H1 to
> zero in its aggregate request-table for all apps.
> - In the mean-while AM gets to need 3 more containers, so a total of 7 including the
4 from previous request.
> - Today, AM sends the absolute number of 7 against H1 to RM as part of its request table.
> - RM seems to be overriding its earlier value of zero against H1 to 7 against H1. And
thus allocating 7 more
> containers.
> - AM already gets 4 in this scheduling iteration, but gets 7 more, a total of 11 instead
of the required 7.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message