hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3508) Preemption processing occuring on the main RM dispatcher
Date Tue, 30 Jun 2015 20:12:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14608986#comment-14608986
] 

Wangda Tan commented on YARN-3508:
----------------------------------

bq. Ok, IIUC you want me to pass RMContext to ProportionalCapacityPreemptionPolicy and get
event handler from there. Correct ?
Yes

bq. Any reason you want this changed?
RM components have a reference to RMContext is a convention, you can see most of RM components
have RMContext. And this can avoid some future code changes if Proportional..Policy needs
more information beyond dispatcher.

bq.  And anyways ProportionalCapacityPreemptionPolicy constructors are used only from tests.
Init is used by production code, I suggest to make constructor/init signature consistent.

> Preemption processing occuring on the main RM dispatcher
> --------------------------------------------------------
>
>                 Key: YARN-3508
>                 URL: https://issues.apache.org/jira/browse/YARN-3508
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager, scheduler
>    Affects Versions: 2.6.0
>            Reporter: Jason Lowe
>            Assignee: Varun Saxena
>         Attachments: YARN-3508.002.patch, YARN-3508.01.patch, YARN-3508.03.patch, YARN-3508.04.patch
>
>
> We recently saw the RM for a large cluster lag far behind on the AsyncDispacher event
queue.  The AsyncDispatcher thread was consistently blocked on the highly-contended CapacityScheduler
lock trying to dispatch preemption-related events for RMContainerPreemptEventDispatcher. 
Preemption processing should occur on the scheduler event dispatcher thread or a separate
thread to avoid delaying the processing of other events in the primary dispatcher queue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message