hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhengchenyu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3091) [Umbrella] Improve and fix locks of RM scheduler
Date Mon, 19 Dec 2016 08:59:59 GMT

    [ https://issues.apache.org/jira/browse/YARN-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15760615#comment-15760615
] 

zhengchenyu commented on YARN-3091:
-----------------------------------

Sorry, although this issue is completed, I still wanna ask for advice.
I think the improvement of performance is not enough, if used read-write lock. Though read-write
lock is not better than synchronized, and solved the problem of dead lock, but it's coarse-grained.
For example, FairScheduler.update() hold a write block, but it still block the FairScheduler.checkAccess(),
we know most code of FairScheduler.update() doesn't matter FairScheduler.checkAccess().
I think we should associate the lock with resources instead of function. For exmaple, demand
is resource of the queue, we could operate it by CAS.
So can we optimize with other way?

> [Umbrella] Improve and fix locks of RM scheduler
> ------------------------------------------------
>
>                 Key: YARN-3091
>                 URL: https://issues.apache.org/jira/browse/YARN-3091
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: capacityscheduler, fairscheduler, resourcemanager, scheduler
>            Reporter: Wangda Tan
>
> In existing YARN RM scheduler, there're some issues of using locks. For example:
> - Many unnecessary synchronized locks, we have seen several cases recently that too frequent
access of scheduler makes scheduler hang. Which could be addressed by using read/write lock.
Components include scheduler, CS queues, apps
> - Some fields not properly locked (Like clusterResource)
> We can address them together in this ticket.
> (More details see comments below)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message