hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-392) Make it possible to schedule to specific nodes without dropping locality
Date Fri, 19 Apr 2013 05:23:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13636077#comment-13636077

Bikas Saha commented on YARN-392:

bq. would it not make sense to go with the convention used in the FS/CS already, in which
the locality delay is a fraction of the cluster size? In your proposal, if I want a node-local
container at node1, would I specify the locality delay on the request for node1 or on the
request for the rack that node1 is on?

Yes. I mentioned in the proposal that the user time interval would be converted to count RM
is using currently by using the total cluster size and NM heartbeat interval. So if cluster
size = C and NM heartbeat every S seconds and user interval is T seconds then wait for (S/T)*C
number of NM heartbeats. So its not any more expensive wrt CPU for the scheduler than what
we currently have.

Specific locality for node would set the delay on the rack and specific locality for ANY would
set the delay on ANY. Isnt that how we set the noAllocateAt flag for the boolean approach?
> Make it possible to schedule to specific nodes without dropping locality
> ------------------------------------------------------------------------
>                 Key: YARN-392
>                 URL: https://issues.apache.org/jira/browse/YARN-392
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Sandy Ryza
>         Attachments: YARN-392-1.patch, YARN-392-2.patch, YARN-392-2.patch, YARN-392.patch
> Currently its not possible to specify scheduling requests for specific nodes and nowhere
else. The RM automatically relaxes locality to rack and * and assigns non-specified machines
to the app.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message