hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3305) AM-Used Resource for leafqueue is wrongly populated if AM ResourceRequest is less than minimumAllocation
Date Tue, 08 Dec 2015 00:18:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-3305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15046033#comment-15046033
] 

ASF GitHub Bot commented on YARN-3305:
--------------------------------------

Github user smarella commented on the pull request:

    https://github.com/apache/hadoop/commit/968425e9f7b850ff9c2ab8ca37a64c3fdbe77dbf#commitcomment-14835835
  
    In hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java:
    In hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java
on line 397:
    YARN-3305 seem to break the Myriad builds (see below). As indicated in YARN-3305, https://issues.apache.org/jira/browse/YARN-3996
is trying to fix the problem. YARN-3996 is currently work-in-progress, but we need to back
port some of it immediately to unblock Myriad. Once YARN-3996 is fully fixed, it needs to
be fully back ported. I've requested Sarjeet to raise two bugs for this (one to provide an
immediate fix and second to back port YARN-3996 once it's resolved).
    
    Currently, the latest Myriad builds are effected by this change. The problem is that,
the AM container resources are normalized during app submission with ```scheduler.getminimumResourceCapability()```,
which comes from yarn.scheduler.minimum-allocation-{mb,vcores,disks}. These values are set
to "0" by Myriad. Hence, the AM container allocated will be of zero size and is killed by
NM as soon as it is launched.
    
    The reason why Myriad needs yarn.scheduler.minimum-allocation-{mb,vcores,disks} to be
0 is for discussed in https://issues.apache.org/jira/browse/MYRIAD-139.


> AM-Used Resource for leafqueue is wrongly populated if AM ResourceRequest is less than
minimumAllocation
> --------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-3305
>                 URL: https://issues.apache.org/jira/browse/YARN-3305
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 2.6.0
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>             Fix For: 2.8.0
>
>         Attachments: 0001-YARN-3305.patch, 0001-YARN-3305.patch, 0002-YARN-3305.patch,
0003-YARN-3305.patch
>
>
> For given any ResourceRequest, {{CS#allocate}} normalizes request to minimumAllocation
if requested memory is less than minimumAllocation.
> But AM-used resource is updated with actual ResourceRequest made by user. This results
in AM container allocation more than Max ApplicationMaster Resource.
> This is because AM-Used is updated with actual ResourceRequest made by user while activating
the applications. But during allocation of container, ResourceRequest is normalized to minimumAllocation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message