hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler
Date Fri, 23 Sep 2016 17:38:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517073#comment-15517073
] 

Arun Suresh commented on YARN-3139:
-----------------------------------

Thanks for the patch [~templedf]..

AbstractYarnScheduler:
# {{AbstractYarnScheduler::getApplicationAttempt()}} : Do you need the read lock, since _applications_
is already a ConcurrentHashMap ?
# {{AbstractYarnScheduler::moveAllApps()}} : Don't think we need a writelock there since the
only state change is affected via an {{RMAppMoveEvent}}. A readLock should suffice, but if
_getAppsInQueue_ is synchronized, even that might not be required.
# Same for the {{AbstractYarnScheduler:: killAllAppsInQueue()}}

SchedulerApplicationAttempt:
# {{SchedulerApplicationAttempt::pendingRelease}} : Maybe use a ConcurrentSkipListSet ?

CapacityScheduler:
# Considering that initScheduler is called exactly once, and the Scheduler cannot be used
before it is initted, do we need a write lock there ?
# Same goes for {{startSchedulerThreads()}} and {{stopService()}}.. [~leftnoteasy] Thoughts
?

FairScheduler:
# In multiple places, maybe it is possible to reduce the granuality of the write locks...
for eg. in {{FairScheduler::addApplication()}}, you dont really need the lock before line
664.
# {{removeApplication()}}, do you need to do a get and then remove... the remove will give
you the previous app and you can log if null right ? Then you wont event need a lock since
*applications* is a concurrent Hashmap.
# In _removeApplicationAttempt_
{code}
    SchedulerApplication<FSAppAttempt> application =
        applications.get(applicationAttemptId.getApplicationId());
    FSAppAttempt attempt = getSchedulerApp(applicationAttemptId);
{code}
can be replaced with {{applications.get(applicationAttemptId.getApplicationId()).getCurrentAppAttempt()}}
right?



> Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler
> ----------------------------------------------------------------------
>
>                 Key: YARN-3139
>                 URL: https://issues.apache.org/jira/browse/YARN-3139
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager, scheduler
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-3139.0.patch, YARN-3139.1.patch, YARN-3139.2.patch
>
>
> Enhance locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler, as mentioned
in YARN-3091, a possible solution is using read/write lock. Other fine-graind locks for specific
purposes / bugs should be addressed in separated tickets.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message