hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-5716) Add global scheduler interface definition and update CapacityScheduler to use it.
Date Sat, 22 Oct 2016 03:59:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15597090#comment-15597090
] 

Sunil G edited comment on YARN-5716 at 10/22/16 3:59 AM:
---------------------------------------------------------

Sorry for pitching in late. 

Patch and approaches looks really great. Thank you. 

Some minor doubts/comments after a first round of scan :)

1. is AppSchedulingInfo#updateMetricsForAllocatedContainer need to be under lock, I think
its safely invoked within a lock block? Do you have some plans to reuse this module later
somewhere, if so its fine.
2. Every time AppSchedulingInfo#getSchedulingPlacementSet is invoked, a new SchedulingPlacementSet
is created with all nodes iterator. Could we have an abstract class here so that we can maintain
this code by keeping a basic impl for other non-used methods?
3. LeafQueue#apply, could we have a more fine grained lock for allocateResource and orderingPolicy.
I think we do not worry about *request* object, correct? pls correct me if I am wrong.
4. Is it possible to keep the interfaces {{SchedulerContainer}} or {{ContainerAllocationProposal}}
more read-only? I think it will help us keep interfaces more cleaner.
5. One doubt in ResourceCommitRequest. Why are we adding allocated/reserved resources to {{totalReleasedResource}}?
Sorry, i didnt understand here.
I think i have only some more minor comments which i ll share in short while. Will make sure
it ll not block the commit process :)



was (Author: sunilg):
Sorry for pitching in late. 

Patch and approaches looks really great. Thank you. 

Some minor doubts/comments after a first round of scan :)

1. is AppSchedulingInfo#updateMetricsForAllocatedContainer need to be under lock, I think
its safely invoked within a lock block? Do you have some plans to reuse this module later
somewhere, if so its fine.
2. Every time AppSchedulingInfo#getSchedulingPlacementSet is invoked, a new SchedulingPlacementSet
is created with all nodes iterator. Could we have an abstract class here so that we can maintain
this code by keeping a basic impl for other non-used methods?
3. LeafQueue#apply, could we have a more fine grained lock for allocateResource and orderingPolicy.
I think we do not worry about *request* object, correct? pls correct me if I am wrong.
4. Is it possible to keep the interfaces {{SchedulerContainer}} or {{ContainerAllocationProposal}}
more read-only? I think it will help us keep interfaces more cleaner.
5. One doubt in ResourceCommitRequest. Why are we adding allocated/reserved resources to {{totalReleasedResource}}?
Sorry, i didnt understand here.
I think i have only some more minor comments which i ll share in short while. Will make sure
it ll block the commit process :)


> Add global scheduler interface definition and update CapacityScheduler to use it.
> ---------------------------------------------------------------------------------
>
>                 Key: YARN-5716
>                 URL: https://issues.apache.org/jira/browse/YARN-5716
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-5716.001.patch, YARN-5716.002.patch, YARN-5716.003.patch, YARN-5716.004.patch,
YARN-5716.005.patch, YARN-5716.006.patch, YARN-5716.007.patch
>
>
> Target of this JIRA:
> - Definition of interfaces / objects which will be used by global scheduling, this will
be shared by different schedulers.
> - Modify CapacityScheduler to use it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message