hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7681) Scheduler should double-check placement constraint before actual allocation is made
Date Tue, 09 Jan 2018 15:17:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16318577#comment-16318577
] 

Weiwei Yang commented on YARN-7681:
-----------------------------------

Hi [~asuresh]

bq. When the processor is turned on, the SchedulingRequest does not flow through the AppPlacementAllocator

Ah, do you mean when processor is turned on, the {{tryCommit}} was called in {{CapacityScheduler#attemptAllocationOnNode}}
with a *false* boolean arg, this way the check in {{AppPlacementAllocator}} code was by passed
in [FiCaSchedulerApp#accept|https://github.com/apache/hadoop/blob/517f2027a5a689fd31ee63f1432441bcdf6ab8f8/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java#L412-L425]
? In this case, I've uploaded 1st patch to add the check per my understanding. Please take
a look, thanks.


> Scheduler should double-check placement constraint before actual allocation is made
> -----------------------------------------------------------------------------------
>
>                 Key: YARN-7681
>                 URL: https://issues.apache.org/jira/browse/YARN-7681
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: RM, scheduler
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>         Attachments: YARN-7681.001.patch
>
>
> This JIRA is created based on the discussions under YARN-7612, see comments after [this
comment|https://issues.apache.org/jira/browse/YARN-7612?focusedCommentId=16303051&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16303051].
AllocationTagsManager maintains tags info that helps to make placement decisions at placement
phase, however tags are changing along with container's lifecycle, so it is possible that
the placement violates the constraints at the scheduling phase. Propose to add an extra check
in the scheduler to make sure constraints are still satisfied during the actual allocation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message