hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrey Stepachev (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-2145) FairScheduler can assign more resources to queue with MaxShare set.
Date Wed, 11 Jun 2014 12:47:02 GMT

     [ https://issues.apache.org/jira/browse/YARN-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrey Stepachev updated YARN-2145:
-----------------------------------

    Description: 
method fair.FSQueue#assignContainerPreCheck() uses Resources.fitsIn incorrectly, because if
queue is at its maximum capacity method returns true and container can be assigned to queue
with maximum resources already consumed.
That is because fitsIn code do '<=' comparison and if allocated resources == max resources
fitsIn still returns true, and consequently pre check successes
{code}
  public static boolean fitsIn(Resource smaller, Resource bigger) {
    return smaller.getMemory() <= bigger.getMemory() &&
        smaller.getVirtualCores() <= bigger.getVirtualCores();
  }
{code}

Provided patch introduced Resources.less method, it more lightweight then lessThan. 

  was:
method fair.FSQueue#assignContainerPreCheck() uses Resources.fitsIn incorrectly, because if
queue is at its maximum capacity method returns true and container can be assigned to queue
with maximum resources already consumed.
That is because fitsIn code do '<=' comparison and if allocated resources == max resources
fitsIn still returns true, and consequently pre check successes
{code}
  public static boolean fitsIn(Resource smaller, Resource bigger) {
    return smaller.getMemory() <= bigger.getMemory() &&
        smaller.getVirtualCores() <= bigger.getVirtualCores();
  }
{code}


> FairScheduler can assign more resources to queue with MaxShare set.
> -------------------------------------------------------------------
>
>                 Key: YARN-2145
>                 URL: https://issues.apache.org/jira/browse/YARN-2145
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>    Affects Versions: 2.5.0
>            Reporter: Andrey Stepachev
>         Attachments: YARN-2145.diff
>
>
> method fair.FSQueue#assignContainerPreCheck() uses Resources.fitsIn incorrectly, because
if queue is at its maximum capacity method returns true and container can be assigned to queue
with maximum resources already consumed.
> That is because fitsIn code do '<=' comparison and if allocated resources == max resources
fitsIn still returns true, and consequently pre check successes
> {code}
>   public static boolean fitsIn(Resource smaller, Resource bigger) {
>     return smaller.getMemory() <= bigger.getMemory() &&
>         smaller.getVirtualCores() <= bigger.getVirtualCores();
>   }
> {code}
> Provided patch introduced Resources.less method, it more lightweight then lessThan. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message