hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrey Stepachev (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2073) Fair Scheduler: Add a utilization threshold to prevent preempting resources when cluster is free
Date Wed, 11 Jun 2014 11:29:02 GMT

    [ https://issues.apache.org/jira/browse/YARN-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14027644#comment-14027644
] 

Andrey Stepachev commented on YARN-2073:
----------------------------------------

I read code and don't cleanly understand why inequality uses Available resources instead of
Allocated?
With current code preemption performed only when cluster utilised under 20percent.
Suppose we have occupied 8GB of 10GB. And default preemption threshold == 0.8f.
>From description I thought that preemption should occur, but it doesn't.
Code below gives us: 
{code}
0.8 < availableMb / maxMem
0.8 < (10-8) / 10
0.8 < 0.2 
false
{code}
{code}
      return (preemptionUtilizationThreshold < Math.max(
          (float) rootMetrics.getAvailableMB() / clusterResource.getMemory(),
          (float) rootMetrics.getAvailableVirtualCores() /
              clusterResource.getVirtualCores()));
    }
{code}

Is that was an intention for this parameter (and it should be documented for non native speakers
to be 
more clear, say: preemption will not performed if utilisation is above this parameter) or
here was a mistake?

> Fair Scheduler: Add a utilization threshold to prevent preempting resources when cluster
is free
> ------------------------------------------------------------------------------------------------
>
>                 Key: YARN-2073
>                 URL: https://issues.apache.org/jira/browse/YARN-2073
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 2.4.0
>            Reporter: Karthik Kambatla
>            Assignee: Karthik Kambatla
>            Priority: Critical
>             Fix For: 2.5.0
>
>         Attachments: yarn-2073-0.patch, yarn-2073-1.patch, yarn-2073-2.patch, yarn-2073-3.patch,
yarn-2073-4.patch
>
>
> Preemption should kick in only when the currently available slots don't match the request.




--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message