hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-752) In AMRMClient, automatically add corresponding rack requests for requested nodes
Date Mon, 10 Jun 2013 00:21:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13679235#comment-13679235

Bikas Saha commented on YARN-752:

Its not 100% clear what the API should be on the AMRMClient for specific requests. A set of
checks need to be made so that unsatisfiable requests cannot be made or illegal mix-n-match
cannot happen. The server side impl is simple and does no checks for the sake of overhead
Regardless, given the protocol API that requires relaxLocality flag to be specified on racks
for the specific nodes, the resolution of nodes to racks is required, even in that case. This
jira is simply for the addition of rack resolution functionality to the AMRMClient. Its current
use is to make sure all nodes are resolved to their racks or else those requests will be unsatisfiable
on the server side, resulting in correctness issues.

The patch by itself has a bug. After a container request has been removed, it should not be
returned via getMatchingRequests() for the inferred rack. i.e. the patch was doing addResourceRequest()
for the inferred rack but not doing the corresponding removeResourceRequest(). The simple
fix is to resolve/dedupe not only during addContainerRequest() but also during removeContainerRequest().
Attaching patch with simple fix and minor changes to the javadoc.
> In AMRMClient, automatically add corresponding rack requests for requested nodes
> --------------------------------------------------------------------------------
>                 Key: YARN-752
>                 URL: https://issues.apache.org/jira/browse/YARN-752
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: api, applications
>    Affects Versions: 2.0.4-alpha
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>         Attachments: YARN-752-1.patch, YARN-752-1.patch, YARN-752-2.patch, YARN-752.3.patch,
> A ContainerRequest that includes node-level requests must also include matching rack-level
requests for the racks that those nodes are on.  When a node is present without its rack,
it makes sense for the client to automatically add the node's rack.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message