hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yufei Gu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6307) Refactor FairShareComparator#compare
Date Mon, 24 Jul 2017 22:28:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099195#comment-16099195
] 

Yufei Gu commented on YARN-6307:
--------------------------------

The overhead of this refactoring would be 
1. More check on variable {{res}}, at most 4 more checking which possible need 4-8 more CPU
instructions, which is fine. 
2. Overhead of function invoking. JVM does inline methods. https://stackoverflow.com/questions/2096361/are-there-inline-functions-in-java

Moreover, this refactor will reduce the computation of fair share comparison, since it does
fair share ratio computation anyway before my patch while it is invoked only if necessary
after my patch. In that sense, I don't think we need to worry about performance. 

> Refactor FairShareComparator#compare
> ------------------------------------
>
>                 Key: YARN-6307
>                 URL: https://issues.apache.org/jira/browse/YARN-6307
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>            Reporter: Yufei Gu
>            Assignee: Yufei Gu
>         Attachments: YARN-6307.001.patch, YARN-6307.002.patch, YARN-6307.003.patch
>
>
> The method does three things: compare the min share usage, compare fair share usage by
checking weight ratio, break tied by submit time and name. They are mixed with each other
which is not easy to read and maintenance, poor style. Additionally, there are potential performance
issues, like no need to check weight ratio if minShare usage comparison already indicate the
order. It is worth to improve considering huge amount invokings in scheduler.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message