hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests
Date Tue, 31 Mar 2015 20:03:02 GMT

    [ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389288#comment-14389288
] 

Wangda Tan commented on YARN-796:
---------------------------------

[~john.jian.fang],
To clarify, now MR doesn't support specifying node labels when submitting jobs. Instead, they
can configure default-node-label-expression in queues which will run their MR jobs to make
all containers allocated in the queue on nodes with specific labels. I'm not sure how you
plan to manage MR jobs in EC2 cluster. Will all MR jobs running in a set if queues? If so,
you can specify configure default-node-label-expression in these queues to get what you want.

And also, if you think default-node-label-expression is not enough, you can file a ticket
under MAPREDUCE to support labels specifying for MR jobs. We can continue discussion to that
ticket.

> Allow for (admin) labels on nodes and resource-requests
> -------------------------------------------------------
>
>                 Key: YARN-796
>                 URL: https://issues.apache.org/jira/browse/YARN-796
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>    Affects Versions: 2.4.1
>            Reporter: Arun C Murthy
>            Assignee: Wangda Tan
>         Attachments: LabelBasedScheduling.pdf, Node-labels-Requirements-Design-doc-V1.pdf,
Node-labels-Requirements-Design-doc-V2.pdf, Non-exclusive-Node-Partition-Design.pdf, YARN-796-Diagram.pdf,
YARN-796.node-label.consolidate.1.patch, YARN-796.node-label.consolidate.10.patch, YARN-796.node-label.consolidate.11.patch,
YARN-796.node-label.consolidate.12.patch, YARN-796.node-label.consolidate.13.patch, YARN-796.node-label.consolidate.14.patch,
YARN-796.node-label.consolidate.2.patch, YARN-796.node-label.consolidate.3.patch, YARN-796.node-label.consolidate.4.patch,
YARN-796.node-label.consolidate.5.patch, YARN-796.node-label.consolidate.6.patch, YARN-796.node-label.consolidate.7.patch,
YARN-796.node-label.consolidate.8.patch, YARN-796.node-label.demo.patch.1, YARN-796.patch,
YARN-796.patch4
>
>
> It will be useful for admins to specify labels for nodes. Examples of labels are OS,
processor architecture etc.
> We should expose these labels and allow applications to specify labels on resource-requests.
> Obviously we need to support admin operations on adding/removing node labels.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message