hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-6216) Unify Container Resizing code paths with Container Updates making it scheduler agnostic
Date Sat, 25 Feb 2017 18:03:44 GMT

     [ https://issues.apache.org/jira/browse/YARN-6216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Arun Suresh updated YARN-6216:
    Attachment: YARN-6216.003.patch

Updating patch based on [~leftnoteasy]'s suggestions.
Thanks for the review Wangda.

I've addressed all your minor concerns
Regarding the major ones:
bq. ... but could potentially make the container-id becomes very sparse. I think this should
be optimizable
I agree, it is a bit dicey though, and I think we should maybe fix it when we hit the issue..
will open another JIRA to address this.

bq. Could you elaborate new changes added to AbstractYarnScheduler#completedContainer?
Moved the code to another method and added docs.

bq.  Can we simplify decrease/demote container process: now...
Based on our offline discussion, we will keep it as such for now, since it would actually
result in a bit more code.

bq. , it increase #live-containers by 1 in scheduler and soon decrease it. Since #live-containers
is visible to client, if changes to fix the issue is not too complex, it's better to optimize
Considering that it is a transient thing (for demote/decresea, it is instantaneuos), if you
don't mind, lets address this in another JIRA if required. 

> Unify Container Resizing code paths with Container Updates making it scheduler agnostic
> ---------------------------------------------------------------------------------------
>                 Key: YARN-6216
>                 URL: https://issues.apache.org/jira/browse/YARN-6216
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler, fairscheduler, resourcemanager
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Arun Suresh
>            Assignee: Arun Suresh
>             Fix For: 3.0.0-alpha3
>         Attachments: YARN-6216.001.patch, YARN-6216.002.patch, YARN-6216.003.patch
> YARN-5959 introduced an {{ContainerUpdateContext}} which can be used to update the ExecutionType
of a container in a scheduler agnostic manner. As mentioned in that JIRA, extending that to
encompass Container resizing is trivial.
> This JIRA proposes to remove all the CapacityScheduler specific code paths. (CapacityScheduler,
CSQueue, FicaSchedulerApp etc.) and modify the code to use the framework introduced in YARN-5959
without any loss of functionality.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message