hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-103) Add a yarn AM - RM client module
Date Wed, 10 Oct 2012 07:23:03 GMT

    [ https://issues.apache.org/jira/browse/YARN-103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473049#comment-13473049
] 

Bikas Saha commented on YARN-103:
---------------------------------

This version of the client is for advanced apps like MR where it lets them avoid duplication
of client boiler plate code + some basic request table inversion logic.
I am open to removing getNumClusterNodes/getAvailableResource helper functions since the values
are available via AllocateResponse. I had put them their because I didnt like that one of
them came from AMResponse and the other from AllocateResponse.
A smarter client will do things like make sure the protocol semantics are being automatically
followed. E.g. in will automatically make sure that the conditions you mention above dont
happen. Also, it will automatically heartbeat with the RM so that the AM does not have to
worry about it. It can even have basic container to task assignment logic. I have a smart
client waiting in the wings that builds on top of this one :P
The test only expects the RM to schedule some containers to it and ack release containers.
So that should be relatively scheduler agnostic. Once we separate functional tests from unit
tests then we could start a single mini cluster for a set of tests and amortize the costs.
I have added a junit test timeout to limit the downside of getting stuck due to a bug.
A custom comparator for ask would have been enough. I have changed that. The previous patch
is the kind of code that comes out of a stuffy nose and head :P
                
> Add a yarn AM - RM client module
> --------------------------------
>
>                 Key: YARN-103
>                 URL: https://issues.apache.org/jira/browse/YARN-103
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Bikas Saha
>            Assignee: Bikas Saha
>         Attachments: YARN-103.1.patch, YARN-103.2.patch, YARN-103.3.patch, YARN-103.4.patch,
YARN-103.4.wrapper.patch
>
>
> Add a basic client wrapper library to the AM RM protocol in order to prevent proliferation
of code being duplicated everywhere. Provide helper functions to perform reverse mapping of
container requests to RM allocation resource request table format.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message