hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5047) Refactor nodeUpdate across schedulers
Date Mon, 30 May 2016 22:19:12 GMT

    [ https://issues.apache.org/jira/browse/YARN-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15306981#comment-15306981

Karthik Kambatla commented on YARN-5047:

Thanks for the update, Ray. Almost there. Comments:
# The following piece is in all schedulers. FifoScheduler has it in nodeUpdate, while the
other two have in attemptScheduling and allocateContainersToNode. We should eventually move
this to nodeUpdate. May be, the right way to do that would be to define an abstract method
called assignContainers(Node) in AbstractYarnScheduler and rename the existing methods in
CS and FS. I am comfortable with doing this in a follow-up JIRA.
    if (rmContext.isWorkPreservingRecoveryEnabled()
        && !rmContext.isSchedulerReadyForAllocatingContainers()) {
# Unlike other schedulers, CS seems to call allocateContainersToNode in handle() instead of
nodeUpdate. May be CS should also override nodeUpdate and call this method there for uniformity
as we refactor things around this? This would in preparation for moving handle() itself to
AbstractYarnScheduler. /cc [~leftnoteasy]
# FifoScheduler seems to explicitly check the node resources are greater than minimumAllocation.
This seems like a good check and something other schedulers should likely do upfront. Another
candidate for this umbrella JIRA? 
# initScheduler has some common code across schedulers. Another JIRA to move the common pieces
to AbstractYarnScheduler? 

> Refactor nodeUpdate across schedulers
> -------------------------------------
>                 Key: YARN-5047
>                 URL: https://issues.apache.org/jira/browse/YARN-5047
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler, fairscheduler, scheduler
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Ray Chiang
>            Assignee: Ray Chiang
>         Attachments: YARN-5047.001.patch, YARN-5047.002.patch, YARN-5047.003.patch
> FairScheduler#nodeUpdate() and CapacityScheduler#nodeUpdate() have a lot of commonality
in their code.  See about refactoring the common parts into AbstractYARNScheduler.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message