hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-3633) With Fair Scheduler, cluster can logjam when there are too many queues
Date Tue, 12 May 2015 17:16:02 GMT
Rohit Agarwal created YARN-3633:
-----------------------------------

             Summary: With Fair Scheduler, cluster can logjam when there are too many queues
                 Key: YARN-3633
                 URL: https://issues.apache.org/jira/browse/YARN-3633
             Project: Hadoop YARN
          Issue Type: Bug
          Components: fairscheduler
    Affects Versions: 2.6.0
            Reporter: Rohit Agarwal
            Assignee: Rohit Agarwal
            Priority: Critical


It's possible to logjam a cluster by submitting many applications at once in different queues.

For example, let's say there is a cluster with 20GB of total memory. Let's say 4 users submit
applications at the same time. The fair share of each queue is 5GB. Let's say that maxAMShare
is 0.5. So, each queue has at most 2.5GB memory for AMs. If all the users requested AMs of
size 3GB - the cluster logjams. Nothing gets scheduled even when 20GB of resources are available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message