hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6144) FairScheduler: preempted resources can become negative
Date Mon, 06 Feb 2017 21:22:41 GMT

    [ https://issues.apache.org/jira/browse/YARN-6144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854760#comment-15854760
] 

Karthik Kambatla commented on YARN-6144:
----------------------------------------

Thanks for finding and working on this, Miklos.

Comments on the patch:
# The debug logging does not add much information. Maybe, we should skip it. Sorry for vacillating
on this. 
# Test: 
## Not all asserts are to identify a regression. That is okay, but I just wanted to call that
out. 
## On each of the asserts, can we use assertEquals instead of assertTrue so test failures
become easier to triage. 
## Can we add a message for each of the new asserts? 
## And, can we use Resources.none() instead of creating a new instance every time? 

> FairScheduler: preempted resources can become negative
> ------------------------------------------------------
>
>                 Key: YARN-6144
>                 URL: https://issues.apache.org/jira/browse/YARN-6144
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler, resourcemanager
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Miklos Szegedi
>            Assignee: Miklos Szegedi
>            Priority: Blocker
>         Attachments: Screen Shot 2017-02-02 at 12.49.14 PM.png, YARN-6144.000.patch,
YARN-6144.001.patch
>
>
> {{preemptContainers()}} calls {{trackContainerForPreemption()}} to collect the list of
containers and resources that were preempted for an application. Later the list is reduced
when {{containerCompleted()}} calls {{untrackContainerForPreemption()}}. The bug is that the
resource variable {{preemptedResources}} is subtracted, not just when the container was preempted
but also when it has completed successfully. This causes that we return an incorrect value
in {{getResourceUsage()}}.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message