hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kannan Rajah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests
Date Fri, 02 Jan 2015 22:11:40 GMT

    [ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14263272#comment-14263272

Kannan Rajah commented on YARN-796:

No, I did mean fair scheduler because that is the one having the load balancing logic. It
does this by sorting the available node capacity and iterating over it one at a time. So with
label based scheduling, the node with maximum available capacity need not be OK for the job.
 I agree that it adds complexity. In fact, we need to add an API to the Scheduler interface
that does not take a Node as input. I will draft a proposal and run it by you to see if that
makes sense. I will also check with Yulia about YARN-2497 and see if this can be done as part
of that.

> Allow for (admin) labels on nodes and resource-requests
> -------------------------------------------------------
>                 Key: YARN-796
>                 URL: https://issues.apache.org/jira/browse/YARN-796
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>    Affects Versions: 2.4.1
>            Reporter: Arun C Murthy
>            Assignee: Wangda Tan
>         Attachments: LabelBasedScheduling.pdf, Node-labels-Requirements-Design-doc-V1.pdf,
Node-labels-Requirements-Design-doc-V2.pdf, YARN-796-Diagram.pdf, YARN-796.node-label.consolidate.1.patch,
YARN-796.node-label.consolidate.10.patch, YARN-796.node-label.consolidate.11.patch, YARN-796.node-label.consolidate.12.patch,
YARN-796.node-label.consolidate.13.patch, YARN-796.node-label.consolidate.14.patch, YARN-796.node-label.consolidate.2.patch,
YARN-796.node-label.consolidate.3.patch, YARN-796.node-label.consolidate.4.patch, YARN-796.node-label.consolidate.5.patch,
YARN-796.node-label.consolidate.6.patch, YARN-796.node-label.consolidate.7.patch, YARN-796.node-label.consolidate.8.patch,
YARN-796.node-label.demo.patch.1, YARN-796.patch, YARN-796.patch4
> It will be useful for admins to specify labels for nodes. Examples of labels are OS,
processor architecture etc.
> We should expose these labels and allow applications to specify labels on resource-requests.
> Obviously we need to support admin operations on adding/removing node labels.

This message was sent by Atlassian JIRA

View raw message