hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "JackZhou (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6516) FairScheduler:the algorithm of assignContainer is so slow for it only can assign a thousand containers per second
Date Wed, 26 Apr 2017 02:15:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-6516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15984016#comment-15984016
] 

JackZhou commented on YARN-6516:
--------------------------------

[~yufeigu] Thank you for you answers!
1.  I was saw it in ganglia for "assigning a container took 3ms" , and it think the metric
is real. Because we have about 80 queues and the two active queue have about 1000 applications.
Use the algorithm to assign a container, it cost 3ms is Certainly need!
2. You are right for "The 39 containers are for AM",  and I am pressure test RM fair scheduler
for improve the performance of schedule. But it is not important for the container is used
for AM or a normal container, it is important  that the resource is very leisure but the scheduler
schedule is so slow.
3. I saw many jira talk about remove continuous scheduling, but I have not saw any report
about the performance decrease when use continuous scheduling.

In a word, I think there is too mush space for optimization  especially in scheduler.


> FairScheduler:the algorithm of assignContainer is so slow for it only can assign a thousand
containers per second
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-6516
>                 URL: https://issues.apache.org/jira/browse/YARN-6516
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: fairscheduler
>    Affects Versions: 2.7.2
>            Reporter: JackZhou
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message