hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tao Jie (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6307) Refactor FairShareComparator#compare
Date Thu, 09 Mar 2017 04:54:38 GMT

    [ https://issues.apache.org/jira/browse/YARN-6307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15902482#comment-15902482
] 

Tao Jie commented on YARN-6307:
-------------------------------

Thank you [~yufeigu], FairShareComparator#compare is called very frequently in each container
allocation process. It would improve the scheduler performance if we can simplify this method.
Furthermore, I don't think it is necessary to sort the queue hierarchy from the root to leafqueue
in every node update. Can we do the sort in the update thread, then share the result for node
update? It would reduce much redundant sort. Maybe we can improve this in another JIRA.

> Refactor FairShareComparator#compare
> ------------------------------------
>
>                 Key: YARN-6307
>                 URL: https://issues.apache.org/jira/browse/YARN-6307
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>            Reporter: Yufei Gu
>            Assignee: Yufei Gu
>
> The method did three things: check the min share ratio, check weight ratio, break tied
by submit time and name. They are mixed with each other which is not easy to read and maintenance,
 poor style. Additionally, there are potential performance issues, like no need to calculate
weight ratio every time. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message