hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1582) Capacity Scheduler: add a maximum-allocation-mb setting per queue
Date Fri, 06 Feb 2015 10:41:41 GMT

    [ https://issues.apache.org/jira/browse/YARN-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14308946#comment-14308946
] 

Hudson commented on YARN-1582:
------------------------------

FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #96 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/96/])
YARN-1582. Capacity Scheduler: add a maximum-allocation-mb setting per queue. Contributed
by Thomas Graves (jlowe: rev 69c8a7f45be5c0aa6787b07f328d74f1e2ba5628)
* hadoop-yarn-project/CHANGES.txt
* hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
* hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacityScheduler.java
* hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerContext.java
* hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
* hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ApplicationMasterService.java
* hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
* hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
* hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
* hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
* hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java
* hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java


> Capacity Scheduler: add a maximum-allocation-mb setting per queue 
> ------------------------------------------------------------------
>
>                 Key: YARN-1582
>                 URL: https://issues.apache.org/jira/browse/YARN-1582
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler
>    Affects Versions: 3.0.0, 0.23.10, 2.2.0
>            Reporter: Thomas Graves
>            Assignee: Thomas Graves
>             Fix For: 2.7.0
>
>         Attachments: YARN-1582-branch-0.23.patch, YARN-1582.002.patch, YARN-1582.003.patch
>
>
> We want to allow certain queues to use larger container sizes while limiting other queues
to smaller container sizes.  Setting it per queue will help prevent abuse, help limit the
impact of reservations, and allow changes in the maximum container size to be rolled out more
easily.
> One reason this is needed is more application types are becoming available on yarn and
certain applications require more memory to run efficiently. While we want to allow for that
we don't want other applications to abuse that and start requesting bigger containers then
what they really need.  
> Note that we could have this based on application type, but that might not be totally
accurate either since for example you might want to allow certain users on MapReduce to use
larger containers, while limiting other users of MapReduce to smaller containers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message