hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lohit Vijayarenu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3214) Add non-exclusive node labels
Date Fri, 27 Mar 2015 05:37:54 GMT

    [ https://issues.apache.org/jira/browse/YARN-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14383364#comment-14383364

Lohit Vijayarenu commented on YARN-3214:

Thanks for comment [~wangda]. As I understand in 2.6 there is already support for multiple
labels on a node, right? If so, as part of this patch (or other patches in same release) are
you planning to both partition and attribute? Going back to having constraint of supporting
only one label per node is already regression. I understand reasoning behind this from capacity
scheduler perspective, but if you are planning to do both partition and attributes as part
of same release, then it is good. Otherwise I am -1 on this approach. 

> Add non-exclusive node labels 
> ------------------------------
>                 Key: YARN-3214
>                 URL: https://issues.apache.org/jira/browse/YARN-3214
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler, resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: Non-exclusive-Node-Partition-Design.pdf
> Currently node labels partition the cluster to some sub-clusters so resources cannot
be shared between partitioned cluster. 
> With the current implementation of node labels we cannot use the cluster optimally and
the throughput of the cluster will suffer.
> We are proposing adding non-exclusive node labels:
> 1. Labeled apps get the preference on Labeled nodes 
> 2. If there is no ask for labeled resources we can assign those nodes to non labeled
> 3. If there is any future ask for those resources , we will preempt the non labeled apps
and give them back to labeled apps.

This message was sent by Atlassian JIRA

View raw message