hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5964) fairscheduler use too many object lock, leads to low performance
Date Thu, 08 Dec 2016 00:25:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15730429#comment-15730429

Karthik Kambatla commented on YARN-5964:

Do you have continuous scheduling turned on? On larger clusters, we have noticed that could
lead to lock contention. 

In any case, I do agree there is need to have more finer grained locks. 

> fairscheduler use too many object lock, leads to low performance
> ----------------------------------------------------------------
>                 Key: YARN-5964
>                 URL: https://issues.apache.org/jira/browse/YARN-5964
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: fairscheduler
>    Affects Versions: 2.7.1
>         Environment: CentOS-7.1
>            Reporter: zhengchenyu
>            Priority: Critical
>             Fix For: 2.7.1
>   Original Estimate: 2m
>  Remaining Estimate: 2m
> When too many applications are running, we found that client couldn't submit the application,
and a high callqueuelength of port 8032. I catch the jstack of resourcemanager when callqueuelength
is too high. I found that the thread "IPC Server handler xxx on 8032" are waitting for the
object lock of FairScheduler, nodeupdate holds the lock of the FairScheduler. Maybe high process
time leads to the phenomenon that client can't submit the application. 
> Here I don't consider the problem that client can't submit the application, only estimates
the performance of the fairscheduler. We can see too many function which needs object lock
are used, the granularity of object lock is too big. For example, nodeUpdate and getAppWeight
wanna hold the same object lock. It is unresonable and inefficiency. I recommand that the
low granularity lock replaces the current lock.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message