hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-9598) Make reservation work well when multi-node enabled
Date Thu, 06 Jun 2019 11:26:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-9598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16857545#comment-16857545
] 

Weiwei Yang commented on YARN-9598:
-----------------------------------

Ping [~jutia], may you help to take a look too? Thanks

> Make reservation work well when multi-node enabled
> --------------------------------------------------
>
>                 Key: YARN-9598
>                 URL: https://issues.apache.org/jira/browse/YARN-9598
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler
>            Reporter: Tao Yang
>            Assignee: Tao Yang
>            Priority: Major
>         Attachments: YARN-9598.001.patch
>
>
> This issue is to solve problems about reservation when multi-node enabled:
>  # As discussed in YARN-9576, re-reservation proposal may be always generated on the
same node and break the scheduling for this app and later apps. I think re-reservation in
unnecessary and we can replace it with LOCALITY_SKIPPED to let scheduler have a chance to
look up follow candidates for this app when multi-node enabled.
>  # Scheduler iterates all nodes and try to allocate for reserved container in LeafQueue#allocateFromReservedContainer.
Here there are two problems:
>  ** The node of reserved container should be taken as candidates instead of all nodes
when calling FiCaSchedulerApp#assignContainers, otherwise later scheduler may generate a reservation-fulfilled
proposal on another node, which will always be rejected in FiCaScheduler#commonCheckContainerAllocation.
>  ** Assignment returned by FiCaSchedulerApp#assignContainers could never be null even
if it's just skipped, it will break the normal scheduling process for this leaf queue because
of the if clause in LeafQueue#assignContainers: "if (null != assignment) \{ return assignment;}"
>  # Nodes which have been reserved should be skipped when iterating candidates in RegularContainerAllocator#allocate,
otherwise scheduler may generate allocation or reservation proposal on these node which will
always be rejected in FiCaScheduler#commonCheckContainerAllocation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message