hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-392) Make it possible to schedule to specific nodes without dropping locality
Date Thu, 09 May 2013 20:27:17 GMT

    [ https://issues.apache.org/jira/browse/YARN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13653143#comment-13653143
] 

Bikas Saha commented on YARN-392:
---------------------------------

I am fine with a boolean because the client can wait for the same timeout and then unset the
flag using a new resource request, if it wants to. We need to have a test that verifies this
behavior.

About the last patch
How about disableAllocation as an alternative name?

Why is there a disable node local request? Node-Specific==(Disable-rack+*). Rack-specific==(Disable-*)
Where does Disable-node make sense?
{code}
+        if (localRequest != null && localRequest.getNoAllocateAt()) {
+          continue;
+        }
{code}

The two checks seem to be different. One checks for containers > 0 while the other does
not. Dont know if that matters in the fair scheduler?
{code}
+        if (rackLocalRequest != null && rackLocalRequest.getNoAllocateAt()) {
+          continue;
+        }
 
         if (rackLocalRequest != null && rackLocalRequest.getNumContainers() != 0
{code}

I dont see rack info being set anywhere. Shouldnt the nodes end up getting rack==default-rack.
If thats true then sending a rack request for rack=rack1 is probably not testing what was
intended, right?
{code}
+    ResourceRequest rackRequest = createResourceRequest(1024, "rack1", 1, 1, true);
{code}

As discussed earlier in this jira, its not possible to mix strict and non-strict allocations
at the same priority. I dont see that being checked/enforced anywhere. Similarly, it does
not look like we can mix strict-node and strict-rack at the same priority.

Are capacity scheduler changes not targeted for this patch?
                
> Make it possible to schedule to specific nodes without dropping locality
> ------------------------------------------------------------------------
>
>                 Key: YARN-392
>                 URL: https://issues.apache.org/jira/browse/YARN-392
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Sandy Ryza
>         Attachments: YARN-392-1.patch, YARN-392-2.patch, YARN-392-2.patch, YARN-392-2.patch,
YARN-392.patch
>
>
> Currently its not possible to specify scheduling requests for specific nodes and nowhere
else. The RM automatically relaxes locality to rack and * and assigns non-specified machines
to the app.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message