hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-926) ContainerManagerProtcol APIs should take in requests for multiple containers
Date Wed, 17 Jul 2013 00:18:49 GMT

    [ https://issues.apache.org/jira/browse/YARN-926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13710520#comment-13710520
] 

Vinod Kumar Vavilapalli commented on YARN-926:
----------------------------------------------

bq. most of the time I would expect that newly avail container allocations for an AM, in a
heartbeat, are scattered through out the cluster with seldom overlaps on the same node.
That is very application specific. I am looking at MR which is as of now the biggest app where
in for all medium to large jobs, we are creating per-container threads inside the AM to launch
them. Having a bulk API simplifies the threading in AM and at the same time helps us to launch
containers faster.

bq. Also, the current response is 'void', just an ACK, what would be the response for multiple
containers.
We do have an empty response object in the protocol, it will become a list. Similarly the
client library needs to be modified - may be we can have both single container and multi-container
APIs in the clients.

bq. Finally, if this ends up being the last standing blocker, we should not block the release
because of it.
There are a couple of blockers that are PA and that need some review love. Won't block release
for this. I'll ask others' help to move this forward in any case.

BTW, I've always wanted to do this, but the freeze for APIs is what triggered me to file the
ticket now. 
                
> ContainerManagerProtcol APIs should take in requests for multiple containers
> ----------------------------------------------------------------------------
>
>                 Key: YARN-926
>                 URL: https://issues.apache.org/jira/browse/YARN-926
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
>            Priority: Blocker
>
> AMs typically have to launch multiple containers on a node and the current single container
APIs aren't helping. We should have all the APIs take in multiple requests and return multiple
responses.
> The client libraries could expose both the single and multi-container requests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message