hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-9200) Enable resource configuration of queue capacity for different resources independently
Date Tue, 15 Jan 2019 17:40:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-9200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743248#comment-16743248

Wangda Tan commented on YARN-9200:

[~aihuaxu], adding percentage to different resource types is what we planned to do but didn't
start, it makes a lot of sense for heterogeneous cluster and queue requirements. [~rohithsharma]
mentioned above this before. [~rohithsharma] is there any Jira filed before? And what's the
state of that Jira now?

> Enable resource configuration of queue capacity for different resources independently
> -------------------------------------------------------------------------------------
>                 Key: YARN-9200
>                 URL: https://issues.apache.org/jira/browse/YARN-9200
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacity scheduler
>    Affects Versions: 3.1.0
>            Reporter: Aihua Xu
>            Assignee: Aihua Xu
>            Priority: Major
> In capacity scheduler, currently two resource allocations are supported. 1. percentage
allocation for child queues -  the child queue gets a defined percentage of the resources
for all the resource types; 2. absolute values  (YARN-5881) - each resource is configured
an absolute values.
> Right now we can't mix these case together and it would also very confusing to mix them
in one cluster. The second case actually is more targeting toward cloud env. 
> In a non-cloud env, the ability to configure each resource independently is also useful,
but percentage is preferable over absolute value. One thought here is to add the percentage
configuration for each resource type on the queue. That would allow us to configure memory
bounded queues, or CPU bounded queues. We can also keep backward compatible: each resource
type just gets the same percentage if no percentage is configured for individual resource

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message