hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashwin Shankar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1913) With Fair Scheduler, cluster can logjam when all resources are consumed by AMs
Date Fri, 30 May 2014 23:28:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14014357#comment-14014357

Ashwin Shankar commented on YARN-1913:

Hey [~sandyr], quick comment 
bq.I think it might make sense to only allow the queue-level maxAMShare on leaf queues for
the moment. I can't think of a strong reason somebody would want to set it on a parent queue
For NestedUserQueue rule, user queues would be created dynamically under a parent. For this
use case,
maxAMShare at the parent would be useful, since leaf user queues are not configured in the
alloc xml. 
I see your point that it would complicate the logic at MaxRunningAppsEnforcer,but just wanted
to bring this up in case you
didn't consider this use case.

> With Fair Scheduler, cluster can logjam when all resources are consumed by AMs
> ------------------------------------------------------------------------------
>                 Key: YARN-1913
>                 URL: https://issues.apache.org/jira/browse/YARN-1913
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 2.3.0
>            Reporter: bc Wong
>            Assignee: Wei Yan
>         Attachments: YARN-1913.patch, YARN-1913.patch, YARN-1913.patch, YARN-1913.patch,
> It's possible to deadlock a cluster by submitting many applications at once, and have
all cluster resources taken up by AMs.
> One solution is for the scheduler to limit resources taken up by AMs, as a percentage
of total cluster resources, via a "maxApplicationMasterShare" config.

This message was sent by Atlassian JIRA

View raw message