hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-7669) [API] Introduce interfaces for placement constraint processing
Date Wed, 20 Dec 2017 07:10:00 GMT

     [ https://issues.apache.org/jira/browse/YARN-7669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Arun Suresh updated YARN-7669:
    Attachment: YARN-7669-YARN-6592.003.patch

Updating patch based on [~kkaranasos]'s suggestions.

bq. RejectionReason: explain what each is expected to do. BTW if all our constraints are soft,
why would we have a could_not_place
I am thinking that should be a property of the Algorithm, not the system. We can have an Algorithm
that assumes softness always and thus will never reject a request. But it should be possible
to plugin a strict implementation of the Algorithm that rejects a SchedulingRequest if it
can find a Node that exactly matches.

I renamed the SchedulingProposalCollector to AlgorithmOutput collector - it seemed to make
more sense.

I also removed the SchedulingRequestHandler and SchedulingResponseHandler. Lets put them in
as and when we need them.

[~cheersyang], Thanks for the review.

bq. ine 55: #getNodes returns list of node locations and whose size equals to number of allocation
in the scheduling request, why not to propose some more nodes than asked in case some of them
get rejected by scheduler?
The PlacedSchedulingRequest is a Scheduling Request for which the Algorithm was able to associate
a node with. For each satisfied numAllocation, the Algorithm should add a Node to the list.
We are splitting the scheduling into essentially 2 phases: the placement phase (where we decide
which node) and an allocation phase (where we try to actually allocate the request to the
Node). The PlacedSchedulingRequest is the output of phase 1 - which means at that point we
have already considered all the possible nodes.

bq. There is no set or add method for placed/rejected requests in this class.
True, I just return the whole collection. The client is free to play with it (Did not want
to complicate the first patch)

bq. It looks like a SchedulingRequest can be either accepted or rejected, if a request asks
for 100 containers and only 1 of them could not be allocated, it will be just simply rejected?
Nope. If the framwork was able to allocate 99, then the response will contain a single Rejected
SchedulingRequest with numAllocations = 1. If you look at v006 and v005 of the YARN-7612 patch,
you can see the full workflow along with testcases. Let me know if it makes sense.

bq. RejectionReason: What's the purpose of this? 
I've updated with some documentation - hopefully that will help clarify somethings. Also please
look at v005 and v006 versions of the YARN-7612 patch. And you can see how it is being used.

> [API] Introduce interfaces for placement constraint processing
> --------------------------------------------------------------
>                 Key: YARN-7669
>                 URL: https://issues.apache.org/jira/browse/YARN-7669
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun Suresh
>            Assignee: Arun Suresh
>         Attachments: YARN-7669-YARN-6592.001.patch, YARN-7669-YARN-6592.002.patch, YARN-7669-YARN-6592.003.patch
> As per discussions in YARN-7612. This JIRA will introduce the generic interfaces which
will be implemented in YARN-7612

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message