hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Feng Yuan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5427) Labeled apps cant use <DEFAULT_PARTITION> resources in 2.8
Date Mon, 25 Jul 2016 12:06:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-5427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15391780#comment-15391780
] 

Feng Yuan commented on YARN-5427:
---------------------------------

thanks for your quickly reply,i agree with your explain about the "All queues can access to
nodes without label". Along this line of thought:
If i set queue A "yarn.scheduler.capacity.root.A.accessible-node-labels"=x  --means it can
access x and default.
and i unset "yarn.scheduler.capacity.root.A.default-node-label-expression" as doc say "By
default, this is empty, so application will get containers from nodes without label." if my
app can get containers from "default" firstly and when default used up gets containers from
x?

In fact, my demand is jobs in queue A(batch data work) can use one partition as 'MR' and another
partition as 'SPARK'(high memory) such as 100%*MR and 30%*SPARK.
and jobs in queue B(ML jobs) can use 70%*SPARK. could i achieve this by now(2.8)?

> Labeled apps cant use <DEFAULT_PARTITION> resources in 2.8
> ----------------------------------------------------------
>
>                 Key: YARN-5427
>                 URL: https://issues.apache.org/jira/browse/YARN-5427
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler
>    Affects Versions: 2.8.0
>            Reporter: Feng Yuan
>             Fix For: 2.8.0
>
>
> Queue A use partition X,When x is used up,app cant get resource from <DEFAULT_PARTITION>.
> I find this behavior is different from 2.7.x.Also i find this in the document of 2.7.2
as blew:
> "All queues can access to nodes without label"--https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeLabel.html
> I think it means if queue A`s partition x is used up,app can get containers from DEFAULT_PARTITION?
Do i misstake?
> And I can confirm my configs is correct.Because i can run profectlly in 2.7.x use this
configation too.
> I have observed this in 2.8: "<DEFAULT_PARTITION> is marked as <ECLUSIVE_PARTITION>"
in webui.Is this the reason? 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message