hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-5716) Add global scheduler interface definition and update CapacityScheduler to use it.
Date Tue, 25 Oct 2016 19:13:59 GMT

     [ https://issues.apache.org/jira/browse/YARN-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Wangda Tan updated YARN-5716:
    Attachment: YARN-5716.009.patch

Thanks [~sunilg] for comments:

For 1), yes it looks like we're doing UL check twice, however, the second time only check
one application instead of thousands of apps.
So the time spent on UL check for accept/apply is ignoreable from my performance tests.

For 2), we will not check allocation-from-reserved container by the outter if (...) check

For 3), Jian asked the same question, see my answer https://issues.apache.org/jira/browse/YARN-5716?focusedCommentId=15593292&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15593292:
bq. No we cannot do this merge, because it is possible that in the previous reservedContainer
!= null if, we reserve a new container so the check is not valid.

For 4):
bq. a reference for resourceRequests is kept and under failure updated back to schedulingInfo
object. I do not feel its a clean implementation... 
Actually we clone the ResourceRequest, and we need to check and make sure ResourceRequest
is not changed or still required by the apps.

bq. Method is very lengthy. We could take out logic like increase request etc. So it ll be
more easier.

bq. readLock is mostly needed for increased request and for appSchedulingInfo. I think some
optimizations here could be done separately in another ticket as improvement.
Yeah I agree, I would prefer to keep it as-is unless we find any performance issues.

bq. preferring equals instead of if (fromReservedContainer != reservedContainerOnNode)
I intended to do that, I want to make sure two reserved container are pointed to the same

Please check ver.9 patch

> Add global scheduler interface definition and update CapacityScheduler to use it.
> ---------------------------------------------------------------------------------
>                 Key: YARN-5716
>                 URL: https://issues.apache.org/jira/browse/YARN-5716
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-5716.001.patch, YARN-5716.002.patch, YARN-5716.003.patch, YARN-5716.004.patch,
YARN-5716.005.patch, YARN-5716.006.patch, YARN-5716.007.patch, YARN-5716.008.patch, YARN-5716.009.patch
> Target of this JIRA:
> - Definition of interfaces / objects which will be used by global scheduling, this will
be shared by different schedulers.
> - Modify CapacityScheduler to use it.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message