hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhengchenyu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6407) Improve and fix locks of RM scheduler
Date Thu, 30 Mar 2017 09:14:41 GMT

    [ https://issues.apache.org/jira/browse/YARN-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948718#comment-15948718
] 

zhengchenyu commented on YARN-6407:
-----------------------------------

[~daniel@cloudera.com]
Yes, I think so. Though this work is hard, I think it's necessary!

> Improve and fix locks of RM scheduler
> -------------------------------------
>
>                 Key: YARN-6407
>                 URL: https://issues.apache.org/jira/browse/YARN-6407
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>    Affects Versions: 2.7.1
>         Environment: CentOS 7, 1 Gigabit Ethernet
>            Reporter: zhengchenyu
>             Fix For: 2.7.1
>
>   Original Estimate: 2m
>  Remaining Estimate: 2m
>
> First´╝îthis issue dose not duplicate the YARN-3091.
> In our cluster, we have 5k nodes, and the server is configured with 1 Gigabit Ethernet.
So network is bottleneck in our cluster.
> We must distcp data from warehouse, because of 1 Gigabit Ethernet, we must set yarn.scheduler.fair.max.assign
to 5, or must lead to hotspot.
> The setting that max.assign is 5 lead to the assigned ability decreased. So we start
the ContinuousSchedulingThread. 
> As more applicaitons running in our cluster , and with ContinuousSchedulingThread, the
problem of lock contention is more serious. 
> In our cluster, the callqueue of ApplicationMasterSeriver's rpc is high occasionally.
we worried that more problem occure in future with more application are running.
> Here is our logical graph:
> "1 Gigabit Ethernet" and "data hot spot" ==> "set yarn.scheduler.fair.max.assign to
5" ==> "ContinuousSchedulingThread is started" and "more applcations" => "lock contention"
> I know YARN-3091 solved this problem, but the patch aims that change the object lock
to read write lock. This change is still Coarse-Grained. So I think we lock the resources
or not lock the large section code.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message