hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian Fang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests
Date Mon, 14 Jul 2014 23:59:08 GMT

    [ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061452#comment-14061452
] 

Jian Fang commented on YARN-796:
--------------------------------

In our environment, most likely the label condition will be OR, not AND. But it is good to
support basic logic such as AND, OR, and NOT.

Users may like to allocate application masters to nodes only with specific labels. This is
a special use case because the AM container is actually launched by hadoop itself, not the
user. You may like to add some parameters such as "yarn.app.mapreduce.am.labels" so that hadoop
will honor this parameter. You may also like to add an option like "yarn.label.enabled" to
turn on and off the label feature.

Why do users have to choose either decentralized or centralized label configuration? The labels
could be static and dynamic. The static ones should be loaded from yarn-site.xml on each node
and the dynamic ones should be specified by a restful API or ADMIN. To me, the restful API
could be more useful than the Admin UI. For example, everything is automated for clusters
in a cloud and no manual work in most cases. As a result, I would rather to have a restful
API to update the labels on a node directly through node manager, which will in return sync
with resource manager. Or the API will update both resource manager and the node manager if
the sync time is a problem here.  





> Allow for (admin) labels on nodes and resource-requests
> -------------------------------------------------------
>
>                 Key: YARN-796
>                 URL: https://issues.apache.org/jira/browse/YARN-796
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun C Murthy
>            Assignee: Wangda Tan
>         Attachments: LabelBasedScheduling.pdf, Node-labels-Requirements-Design-doc-V1.pdf,
YARN-796.patch
>
>
> It will be useful for admins to specify labels for nodes. Examples of labels are OS,
processor architecture etc.
> We should expose these labels and allow applications to specify labels on resource-requests.
> Obviously we need to support admin operations on adding/removing node labels.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message