Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Sat, 16 Feb 2013 01:47:12 +0000 (UTC)
From: "Sandy Ryza (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12631851.1360628156997.296180.1360979232373@arcas>
In-Reply-To: <JIRA.12631851.1360628156997@arcas>
References: <JIRA.12631851.1360628156997@arcas>
Subject: [jira] [Commented] (YARN-392) Make it possible to schedule to
 specific nodes without dropping locality
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579675#comment-13579675 ] 

Sandy Ryza commented on YARN-392:
---------------------------------

I've thought about this further a little.  The alternative that occurs to me would be to have the option to associate an group ID with a resource request.  Under the current model, when a container is assigned, requests are decremented "up", i.e. if it's a node-local container, the requests for the corresponding rack-local container and * are decremented.  This would remain the same, unless the assigned container has a group ID, in which case all other requests with that group ID would be decremented instead.

In some ways, this resembles the task-centric approach proposed in YARN-371, but it avoids most of the performance implications by allowing resource-centric scheduling for applications like mapreduce that don't have these special needs.  The disadvantage of this approach would be that it would require more complicated scheduling logic and data structures to handle the two cases.  The advantage of it is that it would be able to represent node-only requests with multiple nodes, which may be essential for some applications.  It could also be overloaded to handle gang-scheduling.

I'm having trouble coming up with anything that's substantively than these two approaches.  Are there other alternatives I'm missing?

                
> Make it possible to schedule to specific nodes without dropping locality
> ------------------------------------------------------------------------
>
>                 Key: YARN-392
>                 URL: https://issues.apache.org/jira/browse/YARN-392
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Sandy Ryza
>
> Currently its not possible to specify scheduling requests for specific nodes and nowhere else. The RM automatically relaxes locality to rack and * and assigns non-specified machines to the app.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira