hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-392) Make it possible to schedule to specific nodes without dropping locality
Date Fri, 22 Feb 2013 01:33:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13583792#comment-13583792

Sandy Ryza commented on YARN-392:

The proposal of per-app delay-scheduling parameters is one I hadn't thought of, and I think
a good one for many use cases.  Do you mean that the delay threshold would be configurable
per-app or per-priority?

The cases that I don't think it supports are:
* If the delay threshold is only configurable per app, an app needs some containers strictly
on specific nodes, and for other containers only has loose preferences.
* An application wants two containers, the first on only node1 or node2 and the second on
only node3 or node4.  What tells the scheduler not to assign both of the containers on node1
and node2?  These containers could be requested at different priorities, but that would essentially
be using priorities to do task-centric scheduling.

Are these use cases non-goals for YARN?  Correct me if I'm wrong, but my understanding was
that the primary reason that the resource scheduler is not a task scheduler is for performance
reasons.  If we can allow it to be task-centric when necessary, but avoid the performance
impact of making it task-centric all the time, it will support location-specific scheduling
in the most flexible and intuitive way.

I hope this isn't rehashing the debate from YARN-371.  For anybody who will be the YARN meetup
tomorrow, it would be great to chat about this for a couple minutes.
> Make it possible to schedule to specific nodes without dropping locality
> ------------------------------------------------------------------------
>                 Key: YARN-392
>                 URL: https://issues.apache.org/jira/browse/YARN-392
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Sandy Ryza
>         Attachments: YARN-392.patch
> Currently its not possible to specify scheduling requests for specific nodes and nowhere
else. The RM automatically relaxes locality to rack and * and assigns non-specified machines
to the app.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message