hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-392) Make it possible to schedule to specific nodes without dropping locality
Date Sat, 16 Feb 2013 01:47:12 GMT

    [ https://issues.apache.org/jira/browse/YARN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579675#comment-13579675

Sandy Ryza commented on YARN-392:

I've thought about this further a little.  The alternative that occurs to me would be to have
the option to associate an group ID with a resource request.  Under the current model, when
a container is assigned, requests are decremented "up", i.e. if it's a node-local container,
the requests for the corresponding rack-local container and * are decremented.  This would
remain the same, unless the assigned container has a group ID, in which case all other requests
with that group ID would be decremented instead.

In some ways, this resembles the task-centric approach proposed in YARN-371, but it avoids
most of the performance implications by allowing resource-centric scheduling for applications
like mapreduce that don't have these special needs.  The disadvantage of this approach would
be that it would require more complicated scheduling logic and data structures to handle the
two cases.  The advantage of it is that it would be able to represent node-only requests with
multiple nodes, which may be essential for some applications.  It could also be overloaded
to handle gang-scheduling.

I'm having trouble coming up with anything that's substantively than these two approaches.
 Are there other alternatives I'm missing?

> Make it possible to schedule to specific nodes without dropping locality
> ------------------------------------------------------------------------
>                 Key: YARN-392
>                 URL: https://issues.apache.org/jira/browse/YARN-392
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Sandy Ryza
> Currently its not possible to specify scheduling requests for specific nodes and nowhere
else. The RM automatically relaxes locality to rack and * and assigns non-specified machines
to the app.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message