hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler
Date Fri, 23 Sep 2016 18:22:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517172#comment-15517172

Wangda Tan commented on YARN-3139:

Thanks [~asuresh] for the review, actually I'm working on the patch :)

bq. AbstractYarnScheduler::getApplicationAttempt() : Do you need the read lock ...
Agree, fixed

bq. AbstractYarnScheduler::moveAllApps() : Don't think we need a writelock...
The reason I put a writelock because we want the move operation to be serialized, for example,
it may have some issue when we do two moveAll operation at the same time. To me moveApp is
equivalent to remove the app and add the new one. Same as we protect addAppAttempt, we should
use the same writelock for safety.

bq. SchedulerApplicationAttempt::pendingRelease :Maybe use a ConcurrentSkipListSet ?
Unless we need to order the set, I think use ConcurrentHashSet should be enough, SkiplistSet
gets worse performance (O(logN) vs. O(1))

bq. Considering that initScheduler is called exactly once, and the Scheduler cannot be used
before it is initted, do we need a write lock there ?
I would prefer keep it just for safety and consistency, as it doesn't affect performance.
Same for stop and startSchedulerThreads

bq. FairScheduler 2.. 3..

And [~jianhe],
I revisited {{checkSchedContainerChangeRequest}}, it is actually safe, so I removed comment
of the original method.

> Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler
> ----------------------------------------------------------------------
>                 Key: YARN-3139
>                 URL: https://issues.apache.org/jira/browse/YARN-3139
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager, scheduler
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-3139.0.patch, YARN-3139.1.patch, YARN-3139.2.patch
> Enhance locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler, as mentioned
in YARN-3091, a possible solution is using read/write lock. Other fine-graind locks for specific
purposes / bugs should be addressed in separated tickets.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message