hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-752) In AMRMClient, automatically add corresponding rack requests for requested nodes
Date Fri, 07 Jun 2013 21:46:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13678470#comment-13678470

Bikas Saha commented on YARN-752:

The user needs to only specify a node (and may end up getting a container on that node, on
that node's rack or somewhere else. There is no need to specify the rack of a node explicitly
but users may do so. If a user wants any node in a set of racks then they should simply directly
specify those racks.

This code is not quite correct. If nodes n1, n2, n3 (all in rack1) are present then this will
end up calling addResourceRequest 3 times for rack1. Should probably create a local set of
racks, initialized by CR.racks, add nodes' racks to the set. Then call addResourceRequest
for the rack set.
+        String rack = RackResolver.resolve(host).getNetworkLocation();
+        if (rack == null) {
+          LOG.warn("Failed to resolve rack for host " + host + ".");
+        } else if (req.racks == null || !req.racks.contains(rack)) {
+          addResourceRequest(req.priority, rack, req.capability,
+              req.containerCount, req);
+        }

The test needs to improve to catch the above case. Also, should check container count for
the asks to ensure that counts are correct (with or without user specifying the racks). The
test is also calling api's without calling start(). It works now, but will probably break
later once we enforce the life cycle inside the client. Things like rack resolution etc probably
depend on topoplogy information coming from the RM upon registration.
> In AMRMClient, automatically add corresponding rack requests for requested nodes
> --------------------------------------------------------------------------------
>                 Key: YARN-752
>                 URL: https://issues.apache.org/jira/browse/YARN-752
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: api, applications
>    Affects Versions: 2.0.4-alpha
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>         Attachments: YARN-752-1.patch, YARN-752-1.patch, YARN-752.patch
> A ContainerRequest that includes node-level requests must also include matching rack-level
requests for the racks that those nodes are on.  When a node is present without its rack,
it makes sense for the client to automatically add the node's rack.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message