hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yufei Gu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6307) Refactor FairShareComparator#compare
Date Wed, 12 Jul 2017 23:50:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084921#comment-16084921

Yufei Gu commented on YARN-6307:

[~Tao Jie], I like the idea of reduce queue sort frequency, but I don't think it is a good
idea to just sort queues in scheduler update thread.  Resource usages changes during container
assigning, and the sort algorithm(FairShareComparator#compare) heavily relies on resource
usages to meet fairness. In that sense, it's hard to avoid queue sort while assigning container.
BTW, have you filed separate JIRAs for this idea?

> Refactor FairShareComparator#compare
> ------------------------------------
>                 Key: YARN-6307
>                 URL: https://issues.apache.org/jira/browse/YARN-6307
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>            Reporter: Yufei Gu
>            Assignee: Yufei Gu
> The method does three things: check the min share ratio, check weight ratio, break tied
by submit time and name. They are mixed with each other which is not easy to read and maintenance,
 poor style. Additionally, there are potential performance issues, like no need to calculate
weight ratio every time. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message