hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-392) Make it possible to specify hard locality constraints in resource requests
Date Tue, 21 May 2013 19:05:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13663267#comment-13663267
] 

Bikas Saha commented on YARN-392:
---------------------------------

bq. in a followup JIRA we can decide that setting the flag on a node-level request warrants
an exception. Or, in a followup JIRA, we can decide that setting the flag on a node-level
request is supported. Neither of these future changes will be a backwards-incompatible change.
Makes sense to me.

We mean to that level and beyond? e.g. if the flag is set on a rack then it implies locality
wont be relaxed for that rack and * right? Thats what the code in the patch does (continue
if rack is false). OR does the flag need to be set at rack AND * when asking for specific
node?
{code}
+   * For a request at a network hierarchy level, set whether locality can be relaxed
+   * to that level.
{code}

It might help if we write the example in the javadoc by specifying how to use the flag to
enable strict node locality. We dont have code checks about what it legal wrt this flag but
we should spell them out in the javadoc.
{code}
+   * 
+   * For example, if the flag is off on a rack-level <code>ResourceRequest</code>,
+   * containers at that request's priority will not be assigned to nodes on that
+   * request's rack unless requests specifically for those nodes have also been
+   * submitted.
+   * 
+   * If the flag is off on an any-level <code>ResourceRequest</code>, containers
at
+   * that request's priority will only be assigned on racks for which specific
+   * requests have also been submitted.
{code}

My grey cells are not what they used to be :) Can you please help me grok the logic following
statement?
{code}
+    return anyRequest.getNumContainers() > 0 &&
+        (nodeRequest == null || nodeRequest.getRelaxLocality()) &&
+        (anyRequest.getRelaxLocality() ||
+            (rackRequest != null && rackRequest.getNumContainers() > 0)) &&
+        (rackRequest == null || rackRequest.getRelaxLocality() ||
+            (nodeRequest != null && nodeRequest.getNumContainers() > 0)) &&
         Resources.lessThanOrEqual(RESOURCE_CALCULATOR, null,
-            request.getCapability(), node.getRMNode().getTotalCapability());
+            anyRequest.getCapability(), node.getRMNode().getTotalCapability());
{code}

You mean node2 right?
{code}
+    // then node1 should get the container
+    scheduler.handle(node2UpdateEvent);
+    assertEquals(1, app.getLiveContainers().size());
{code}
                
> Make it possible to specify hard locality constraints in resource requests
> --------------------------------------------------------------------------
>
>                 Key: YARN-392
>                 URL: https://issues.apache.org/jira/browse/YARN-392
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Sandy Ryza
>         Attachments: YARN-392-1.patch, YARN-392-2.patch, YARN-392-2.patch, YARN-392-2.patch,
YARN-392-3.patch, YARN-392-4.patch, YARN-392-5.patch, YARN-392-6.patch, YARN-392.patch
>
>
> Currently its not possible to specify scheduling requests for specific nodes and nowhere
else. The RM automatically relaxes locality to rack and * and assigns non-specified machines
to the app.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message