hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-392) Make it possible to schedule to specific nodes without dropping locality
Date Thu, 21 Feb 2013 23:12:15 GMT

    [ https://issues.apache.org/jira/browse/YARN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13583651#comment-13583651
] 

Alejandro Abdelnur commented on YARN-392:
-----------------------------------------

I think the 2nd approach (introducing an ID in the ResourceRequest) is preferable to the boolean
flag as it will allow to express a set of OR localities. 

I've been thinking a bit on how adding/handling this new piece of information would affect
the RM/Scheduler. 

After running some use cases on the whiteboard, I think handling this ID is entirely optional
to the scheduler to implement, the scheduler could completely ignore this ID and things would
still work correctly. In such scenario, an AM would just receive additional container allocations
which in turn it  would return them back to the RM without using them.

A scheduler implementing this ID tracking functionality it would require to have an additional
crossrefence Map keying off pending requests with AppId+ID to be able to access and decrement
the corresponding outstanding resource request. And this does not seem overly complicated.

One small change I would suggest is the resource allocation response to the AM to include
the ID that is being satisfied (in case the scheduler handles IDs). This will help the AM
to do a more optimal decision and bookkeeping on what to use the received allocation for.
                
> Make it possible to schedule to specific nodes without dropping locality
> ------------------------------------------------------------------------
>
>                 Key: YARN-392
>                 URL: https://issues.apache.org/jira/browse/YARN-392
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Sandy Ryza
>         Attachments: YARN-392.patch
>
>
> Currently its not possible to specify scheduling requests for specific nodes and nowhere
else. The RM automatically relaxes locality to rack and * and assigns non-specified machines
to the app.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message