hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5047) Refactor nodeUpdate across schedulers
Date Thu, 19 May 2016 03:38:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290416#comment-15290416
] 

Karthik Kambatla commented on YARN-5047:
----------------------------------------

Excited to see progress here. Thanks for working on this, [~rchiang]. 

Comments: 
# I wonder what the @Lock annotation on getNode does? Looking at the {{@interface Lock}}
# {{AbstractYarnScheduler#nodeUpdate}}:
## While we are formatting, we should probably move the definition of {{releaseResources}}
closer to its use at {{realeasedContainers}}, and may be also renamed to "released" resources?

## Java 7 and beyond, we don't can emit the type when calling constructor. Definitions of
newlyLaunchedContainers and completedContainers
## Is the TODO still required? If yes, mind filing a follow-up JIRA and annotating that TODO
## Shouldn't nodeUpdateInternal be called right at the end? The log message says we are looking
at node for allocations. And, the scheduling code might want to use the latest info on aggregated
containers' and node utilization. YARN-1011 does this. 
## Should we mark this final so it can't be overriden? I see that FairScheduler overrides
this to capture duration of node update. I don't remember the details clearly; are we primarily
interested in the duration to schedule or processing the entire node heartbeat? 
# There is interest to track {{SchedulerHealth}} for FairScheduler as well. Can we file a
follow-up JIRA here to move the corresponding update code also to AbstractYarnScheduler so
we can reuse it in FairScheduler? 
# Why do we need the change to FiCaSchedulerApp? Looks like this comes from {{FifoScheduler#nodeUpdateInternal}}.
Shouldn't the type of getNode there be FiCaSchedulerNode? 

> Refactor nodeUpdate across schedulers
> -------------------------------------
>
>                 Key: YARN-5047
>                 URL: https://issues.apache.org/jira/browse/YARN-5047
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler, fairscheduler, scheduler
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Ray Chiang
>            Assignee: Ray Chiang
>         Attachments: YARN-5047.001.patch, YARN-5047.002.patch
>
>
> FairScheduler#nodeUpdate() and CapacityScheduler#nodeUpdate() have a lot of commonality
in their code.  See about refactoring the common parts into AbstractYARNScheduler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message