hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4866) FairScheduler: AMs can consume all vcores leading to a livelock when using FAIR policy
Date Wed, 25 May 2016 14:22:12 GMT

    [ https://issues.apache.org/jira/browse/YARN-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15300124#comment-15300124
] 

Karthik Kambatla commented on YARN-4866:
----------------------------------------

bq. The parameter queue is used for recursive checking. It will be there if we put this function
in FSQueue. In that case, no obvious reason to move the function to FSQueue, unless FSParentQueue
need it in the future.
Not sure I follow. Would the following not work when implemented in FSQueue? 
{code}
boolean isVcoresOverMaxResource(Resource resource) {
  if (resource.getVirtualCores() > 
        scheduler.getAllocationConfiguration()
             .getMaxResources(getName()).getVirtualCores()) {
     return true;
  }

  FSQueue parent = getParent();
  if (parent == null) {
    return false;
  }

  return parent().isVcoresOverMaxResource();
}
{code}

> FairScheduler: AMs can consume all vcores leading to a livelock when using FAIR policy
> --------------------------------------------------------------------------------------
>
>                 Key: YARN-4866
>                 URL: https://issues.apache.org/jira/browse/YARN-4866
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>    Affects Versions: 2.8.0
>            Reporter: Karthik Kambatla
>            Assignee: Yufei Gu
>         Attachments: YARN-4866.001.patch, YARN-4866.002.patch, YARN-4866.003.patch, YARN-4866.004.patch,
YARN-4866.005.patch, YARN-4866.006.patch, YARN-4866.007.patch
>
>
> The maxAMShare uses the queue's policy for enforcing limits. When using FAIR policy,
this considers only memory. If there are fewer vcores on the cluster, the AMs can end up taking
all the vcores leading to a livelock. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message