hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "MENG DING (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1509) Make AMRMClient support send increase container request and get increased/decreased containers
Date Fri, 16 Oct 2015 16:00:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14960927#comment-14960927

MENG DING commented on YARN-1509:

Hi, [~bikassaha]

Thanks for the comments.

I probably didn't make myself clear. We are on the SAME page that for the sake of point 2
alone, it already makes sense to combine increase/decrease API into one change API: {code}public
abstract void requestContainerResourceChange(Container container, Resource capability);{code}

What I was trying to say in the previous post is that to support mix of increase and decrease
in one change request (point 1) doesn't seem to be very feasible (even at a later date). But
I don't think we need to worry about that for now.

Since we are combining increase/decrease API, we definitely should combine the callback methods
into one as well: onContainerResourceChanged(). At this point, I am inclined to simply do
the following which doesn't incur much code changes. I will discuss further with [~leftnoteasy]
on this. {code}public abstract void onContainersResourceChanged(List<Container> containers);{code}

bq. Or are you saying that invalid container resource change requests are immediately rejected
by the RM synchronously in the allocate RPC?
Yes, the ApplicationMasterService will perform a series of sanity checks (e.g., requested
resource <= maximum allocation, etc), and reject invalid requests immediately. This is
the same for other requests too.

bq. Having a simple cancel request regardless of increase or decrease is preferable since
then we are not leaking the current state of the implementation to the user. It is future
Make sense to me. We can probably have something like {{cancelContainerResourceChange(Container
container)}}, which applies to the container that has an outstanding pending increase sitting
in the pendingChange map. There is no explicit protocol to support cancellation of resource
change yet. For now we can achieve that by issuing a backend decrease request with the target
resource set to the same as the current resource, which effectively cancels any outstanding
increase request. 

> Make AMRMClient support send increase container request and get increased/decreased containers
> ----------------------------------------------------------------------------------------------
>                 Key: YARN-1509
>                 URL: https://issues.apache.org/jira/browse/YARN-1509
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Wangda Tan (No longer used)
>            Assignee: MENG DING
>         Attachments: YARN-1509.1.patch, YARN-1509.2.patch, YARN-1509.3.patch, YARN-1509.4.patch,
> As described in YARN-1197, we need add API in AMRMClient to support
> 1) Add increase request
> 2) Can get successfully increased/decreased containers from RM

This message was sent by Atlassian JIRA

View raw message