hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-927) Change ContainerRequest to not have more than 1 container count and remove StoreContainerRequest
Date Tue, 16 Jul 2013 02:18:49 GMT

    [ https://issues.apache.org/jira/browse/YARN-927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13709380#comment-13709380
] 

Bikas Saha commented on YARN-927:
---------------------------------

IMO calling 
addContainerRequest(new ContainerRequest(5)) 
is only a little bit less work than calling
for(int i=0; i<5; ++i) {addContainerRequest(new ContainerRequest(1));}
Other than that there should be no change in any of those applications. They should continue
to work as is after that.

Book-keeping for store containers is next to impossible when add(pri1, 5), add(pri1, 4) is
called followed remove(pri1, 2). Internally, we dont know whether to remove from the first
CR or the second. That makes getting getMatchingRequest API a non-starter. That is why getMatchingRequest
is restricted for StoredContainerRequest. It makes the API confusing. Allowing users to get
some CR and changing its container count outside of the AMRMClient will lead to other correctness
issues. The API is messy the way it is right now. I had always wanted to do this but lost
track of it because of TEZ stabilization work. Reviewing YARN-521 recently reminded of this
when I noticed the duplication of stuff between CR and StoredCR and how its easy to miss them.

>From what I see, the downside of this very minimal and upside is a much cleaner API. Hence
I want to get this in before beta. I held back on it since YARN-521 was close and didnt want
to cause unnecessary massive merge conflicts because of this simple refactor.
                
> Change ContainerRequest to not have more than 1 container count and remove StoreContainerRequest
> ------------------------------------------------------------------------------------------------
>
>                 Key: YARN-927
>                 URL: https://issues.apache.org/jira/browse/YARN-927
>             Project: Hadoop YARN
>          Issue Type: Task
>            Reporter: Bikas Saha
>            Assignee: Bikas Saha
>
> The downside is having to use more than 1 container request when requesting more than
1 container at * priority. For most other use cases that have specific locations we anyways
need to make multiple container requests. This will also remove unnecessary duplication caused
by StoredContainerRequest. It will make the getMatchingRequest() always available and easy
to use removeContainerRequest().

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message