hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1913) With Fair Scheduler, cluster can logjam when all resources are consumed by AMs
Date Sat, 31 May 2014 17:16:03 GMT

    [ https://issues.apache.org/jira/browse/YARN-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14014716#comment-14014716

Sandy Ryza commented on YARN-1913:

The primary benefit of the logic in MaxRunningAppsEnforcer is that it allows us to enforce
maxRunningApps constraints from queues higher up in the hierarchy, and integrate these with
user maxRunningApps constraints

As we won't have these issues for queue maxAMShares, I think we can avoid touching MaxRunningAppsEnforcer
entirely and just do the checking inside AppSchedulable.assignContainer or FSLeafQueue.assignContainer.

> With Fair Scheduler, cluster can logjam when all resources are consumed by AMs
> ------------------------------------------------------------------------------
>                 Key: YARN-1913
>                 URL: https://issues.apache.org/jira/browse/YARN-1913
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 2.3.0
>            Reporter: bc Wong
>            Assignee: Wei Yan
>         Attachments: YARN-1913.patch, YARN-1913.patch, YARN-1913.patch, YARN-1913.patch,
YARN-1913.patch, YARN-1913.patch
> It's possible to deadlock a cluster by submitting many applications at once, and have
all cluster resources taken up by AMs.
> One solution is for the scheduler to limit resources taken up by AMs, as a percentage
of total cluster resources, via a "maxApplicationMasterShare" config.

This message was sent by Atlassian JIRA

View raw message