hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manikandan R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3153) Capacity Scheduler max AM resource limit for queues is defined as percentage but used as ratio
Date Wed, 31 Jan 2018 16:38:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-3153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347144#comment-16347144
] 

Manikandan R commented on YARN-3153:
------------------------------------

[~leftnoteasy] [~sunilg]

I am interested in working on this. I like proposal mentioned in https://issues.apache.org/jira/browse/YARN-3153?focusedCommentId=14317255&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14317255
except the name. Instead of {{yarn.scheduler.capacity.maximum-am-capacity-per-queue}}, we
can use {{yarn.scheduler.capacity.maximum-am-capacity}} like mentioned in [~cwelch] comment
as anyway this property is meant for queue only. 

Reg inheritance, We can make it similar to "disable_preemption" property. 

1. For "root" queue, default value could be DEFAULT_MAXIMUM_APPLICATIONMASTERS_RESOURCE_CAPACITY
and can be used if value is not configured at root level.
2. Whereas, for "leaf"' queues, default value could be o/p of #1 and can be used if value
is not configured at leaf queue level.

Please share your thoughts.

> Capacity Scheduler max AM resource limit for queues is defined as percentage but used
as ratio
> ----------------------------------------------------------------------------------------------
>
>                 Key: YARN-3153
>                 URL: https://issues.apache.org/jira/browse/YARN-3153
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>            Priority: Critical
>
> In existing Capacity Scheduler, it can limit max applications running within a queue.
The config is yarn.scheduler.capacity.maximum-am-resource-percent, but actually, it is used
as "ratio", in implementation, it assumes input will be \[0,1\]. So now user can specify it
up to 100, which makes AM can use 100x of queue capacity. We should fix that.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message