hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests
Date Thu, 13 Jun 2013 15:11:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13682323#comment-13682323

Alejandro Abdelnur commented on YARN-796:

One thing I forgot to mention before is that labels seem to make sense if the resource-requests
location is ANY or for a rack, for resource-requests that are host specific it does not make
sense. We would have verify then that aspect of a resource-request on arrival to the RM.

I guess something this would enable are resource-request like [(location=node1),(location=rack1),(location=ANY,label=wired-to-switch1)].
The topology would be node1 is in rack1 and rack1 is connected to switch-1. The request says,
I prefer node1, then rack1 and then a node in another rack if that rack is connected to switch-1.

A bit more on modeling them as a resource (I have not thought it in full, so just an idea
to think).

Labels are resources which max=1 and a total that is equal to the max number of containers
a node may have (driven by mem or cpu, whichever allows less containers). Then a node that
has a given label-resource always has capability for it if there is enough capability of memory
and/or CPU. Then, a DRF scheduler could use label-resources for 'fair' allocation decisions.

Another option, away from the resources modeling, would be to model labels similar to location.

Said all this, I think this is a great idea that opens a new set of allocation possibilities.
We should spend some time defining exactly what functionality we want to achieve and then
decide if we do that in incremental phases.

> Allow for (admin) labels on nodes and resource-requests
> -------------------------------------------------------
>                 Key: YARN-796
>                 URL: https://issues.apache.org/jira/browse/YARN-796
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
> It will be useful for admins to specify labels for nodes. Examples of labels are OS,
processor architecture etc.
> We should expose these labels and allow applications to specify labels on resource-requests.
> Obviously we need to support admin operations on adding/removing node labels.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message