hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-7681) Scheduler should double-check placement constraint before actual allocation is made
Date Wed, 10 Jan 2018 08:19:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16319876#comment-16319876
] 

Arun Suresh edited comment on YARN-7681 at 1/10/18 8:18 AM:
------------------------------------------------------------

Apologize for the last minute comment :)
[~cheersyang], actually, was just wondering, instead of passing the allocationTags down to
the ResouceCommitRequest, and performing the check in the FicaSchedulerApp, can we not do
this check inside the CapacityScheduler#attemptAllocationOnNode method, before we do the tryCommit
? I somehow feel this should be something the Scheduler does, and should not concern the app
implementation.

Although your patch should work functionally, I feel it is odd that we use the checkPending
flag to decide if we should perform the {{PlacementConstraintsUtil.canSatisfyConstraints}}
check.


was (Author: asuresh):
Apologize for the last minute comment :)
[~cheersyang], actually, was just wondering, instead of passing the allocationTags down to
the ResouceCommitRequest, and performing the check in the FicaSchedulerApp, can we not do
this check inside the CapacityScheduler#attemptAllocationOnNode method, before we do the tryCommit
? I somehow feel this should be something the Scheduler does, and should concern the app implementation.

Although your patch should work functionally, I feel it is odd that we use the checkPending
flag to decide if we should perform the {{PlacementConstraintsUtil.canSatisfyConstraints}}
check.

> Scheduler should double-check placement constraint before actual allocation is made
> -----------------------------------------------------------------------------------
>
>                 Key: YARN-7681
>                 URL: https://issues.apache.org/jira/browse/YARN-7681
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: RM, scheduler
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>         Attachments: YARN-7681-YARN-6592.001.patch, YARN-7681-YARN-6592.002.patch
>
>
> This JIRA is created based on the discussions under YARN-7612, see comments after [this
comment|https://issues.apache.org/jira/browse/YARN-7612?focusedCommentId=16303051&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16303051].
AllocationTagsManager maintains tags info that helps to make placement decisions at placement
phase, however tags are changing along with container's lifecycle, so it is possible that
the placement violates the constraints at the scheduling phase. Propose to add an extra check
in the scheduler to make sure constraints are still satisfied during the actual allocation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message