hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2694) Ensure only single node labels specified in resource request / host, and node label expression only specified when resourceName=ANY
Date Mon, 19 Jun 2017 03:07:02 GMT

    [ https://issues.apache.org/jira/browse/YARN-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053428#comment-16053428
] 

Naganarasimha G R commented on YARN-2694:
-----------------------------------------

Hi [~zhengchenyu], 
    Currently labels are viewed as a partition or pool where resource guarantee is ensured
among the queues who have the access to it.
So say cluster has a single node and  node has labels x,y. There exists 2 queues Q1 and Q2,
and for x label guarantees are Q1(25) & Q2(75), for y label guarantees are Q1(50) &
Q2(50), suppose for some app in *Q1* requesting for label *x* is already allocated 25% and
now some other app in *Q1* is requesting for label *y* then do you need to allocate this node
 to the app in Q1 ? if you allocate then guarantees for label *x* for the *Q2* will not be
met.
Practically a node can belong to only one partition when we want to ensure some policy of
resource sharing among the queues (irrespective of FAIR/CS ). Hence the above modification
was done. To support multiple labels(precisely node attributes) we have raised YARN-3409.
May be more can be discussed there.

> Ensure only single node labels specified in resource request / host, and node label expression
only specified when resourceName=ANY
> -----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-2694
>                 URL: https://issues.apache.org/jira/browse/YARN-2694
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler, resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>              Labels: 2.6.1-candidate
>             Fix For: 2.7.0, 2.6.1, 3.0.0-alpha1
>
>         Attachments: YARN-2694-20141020-1.patch, YARN-2694-20141021-1.patch, YARN-2694-20141023-1.patch,
YARN-2694-20141023-2.patch, YARN-2694-20141101-1.patch, YARN-2694-20141101-2.patch, YARN-2694-20150121-1.patch,
YARN-2694-20150122-1.patch, YARN-2694-20150202-1.patch, YARN-2694-20150203-1.patch, YARN-2694-20150203-2.patch,
YARN-2694-20150204-1.patch, YARN-2694-20150205-1.patch, YARN-2694-20150205-2.patch, YARN-2694-20150205-3.patch,
YARN-2694-branch-2.6.1.txt
>
>
> Currently, node label expression supporting in capacity scheduler is partial completed.
Now node label expression specified in Resource Request will only respected when it specified
at ANY level. And a ResourceRequest/host with multiple node labels will make user limit, etc.
computation becomes more tricky.
> Now we need temporarily disable them, changes include,
> - AMRMClient
> - ApplicationMasterService
> - RMAdminCLI
> - CommonNodeLabelsManager



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message