hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantinos Karanasos (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7774) Miscellaneous fixes to the PlacementProcessor
Date Fri, 19 Jan 2018 09:17:03 GMT

    [ https://issues.apache.org/jira/browse/YARN-7774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331969#comment-16331969

Konstantinos Karanasos commented on YARN-7774:

Thanks [~asuresh] for the patch. Looks good overall, some comments:
 * Do we really need the CircularIterator? It seems to me that you can have a normal iterator
initialized outside the for loop and then each time hasNext()=false, you can re-initialize
it. But maybe I am missing something.
 * For what [~cheersyang] mentioned about not being affinity friendly, you could make a check
whether the constraint has minCardinality>0 and scope=NODE, and then keep the iterator
at the same place in such a case. But if you feel it is an over-optimization for the time
being, I am fine tackling it in another JIRA. Up to you.
 * Do we clean up the black list for each tag? It seems that black-listing can change based
on the allocations that have been done so far, so we might need to use it carefully.

> Miscellaneous fixes to the PlacementProcessor
> ---------------------------------------------
>                 Key: YARN-7774
>                 URL: https://issues.apache.org/jira/browse/YARN-7774
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun Suresh
>            Assignee: Arun Suresh
>            Priority: Blocker
>         Attachments: YARN-7774-YARN-6592.001.patch, YARN-7774-YARN-6592.002.patch, YARN-7774-YARN-6592.003.patch,
> JIRA to track the following minor changes:
> * Scheduler must normalize requests that are made using the {{attemptAllocationOnNode}}
> * Currently, the placement algorithm resets the node iterator for each request. The Placement
Algorithm should either shuffle the node iterator OR use a circular iterator - to ensure a)
more nodes are looked at and b) bias against placing too many containers on the same node
> * Add a placement retry loop for rejected requests - since there are cases especially,
when Constraints will be satisfied after a subsequent request has been placed.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message