hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantinos Karanasos (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7437) Give SchedulingPlacementSet to a better name.
Date Fri, 03 Nov 2017 22:04:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16238461#comment-16238461

Konstantinos Karanasos commented on YARN-7437:

Thanks for starting this, [~leftnoteasy].

Given that we are at it, I think the {{PlacementSet}} class is confusing too, both because
it is essentially an iterator over multiple nodes and because it has very similar name with
{{SchedulingPlacementSet}} although they do something very different.

So, let's see what is the current functionality first -- correct me if I am wrong.
There is one {{SchedulingPlacementSet}} per application, storing the outstanding resource/scheduling
requests. This class is also doing the allocation, given a particular node. So, in the current
implementation, we do locality, thus we have the allocateNodeLocal, rackLocal, etc.
At the same time, I see the {{getPreferredNodeIterator()}}, which is capable of getting a
list of nodes instead of a single node. As we discussed, this is currently being used with
a single node.
[offtopic: BTW (not for this JIRA that is about renaming, but since I am looking at this code),
I don't see why we call the getPreferredNodeIterator outside the SchedulingPlacementSet, I
would expect this logic to happen within the SchedulingPlacementSet. As in, you give it multiple
nodes, and then it decides what is the right one, based on its implementation. For example,
you have locality placement, you give it a bunch of nodes, then internally the SchedulingPlacementSet
decides which is the best of these nodes to use for each outstanding request. It does not
matter /offtopic]

So, on the renaming:
- I think we should rename the {{SchedulingSet}} to something like {{CandidateNodeSet}}, or
anything that has {{NodeSet}} or {{NodeSetIterator}} in it.
- For the {{SchedulingPlacementSet}}, I think we should have in the name something about the
{{App}}, like {{AppPlacementXYZ}} or {{AppSchedulingXYZ}}, where XYZ can be Strategy, although
it is not just a strategy, because it does the accounting for outstanding requests too.

> Give SchedulingPlacementSet to a better name.
> ---------------------------------------------
>                 Key: YARN-7437
>                 URL: https://issues.apache.org/jira/browse/YARN-7437
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>            Priority: Major
> Currently, the SchedulingPlacementSet is very confusing. Here're its responsibilities:
> 1) Store ResourceRequests. (Or SchedulingRequest after YARN-6592).
> 2) Decide order of nodes to allocate when there're multiple node candidates.
> 3) Decide if we should reject node for given requests.
> 4) Store any states/cache can help make decision for #2/#3

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message