hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1913) With Fair Scheduler, cluster can logjam when all resources are consumed by AMs
Date Wed, 28 May 2014 21:07:02 GMT

    [ https://issues.apache.org/jira/browse/YARN-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14011587#comment-14011587
] 

Sandy Ryza commented on YARN-1913:
----------------------------------

I think we should avoid doing approximate calculation through the minimum allocation.  We
need to handle situations where AM resources are much larger than the min, and situations
where the minimum allocation will be 0 (common on Llama-enabled clusters).

This would have the added benefit of avoiding touching the "runnability" machinery, which
is already bordering on over-complicated.

> With Fair Scheduler, cluster can logjam when all resources are consumed by AMs
> ------------------------------------------------------------------------------
>
>                 Key: YARN-1913
>                 URL: https://issues.apache.org/jira/browse/YARN-1913
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 2.3.0
>            Reporter: bc Wong
>            Assignee: Wei Yan
>              Labels: easyfix
>         Attachments: YARN-1913.patch, YARN-1913.patch, YARN-1913.patch, YARN-1913.patch
>
>
> It's possible to deadlock a cluster by submitting many applications at once, and have
all cluster resources taken up by AMs.
> One solution is for the scheduler to limit resources taken up by AMs, as a percentage
of total cluster resources, via a "maxApplicationMasterShare" config.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message