hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei Yan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-1913) With Fair Scheduler, cluster can logjam when all resources are consumed by AMs
Date Thu, 22 May 2014 01:04:33 GMT

     [ https://issues.apache.org/jira/browse/YARN-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Wei Yan updated YARN-1913:
--------------------------

    Attachment: YARN-1913.patch

init patch for review.
Add queueMaxAMShare configuration for each queue. And update the code in MaxRunningAppsEnforcer.java
by considering AM share. Instead of using accurate AM resource usage, here use an easier way.
The max_app_limited_by_AM = (queue.queueMaxAMShare * queue.maxShare) / scheduler.minAllocation.

> With Fair Scheduler, cluster can logjam when all resources are consumed by AMs
> ------------------------------------------------------------------------------
>
>                 Key: YARN-1913
>                 URL: https://issues.apache.org/jira/browse/YARN-1913
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 2.3.0
>            Reporter: bc Wong
>            Assignee: Karthik Kambatla
>         Attachments: YARN-1913.patch
>
>
> It's possible to deadlock a cluster by submitting many applications at once, and have
all cluster resources taken up by AMs.
> One solution is for the scheduler to limit resources taken up by AMs, as a percentage
of total cluster resources, via a "maxApplicationMasterShare" config.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message