hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xianyin Xin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3547) FairScheduler: Apps that have no resource demand should not participate scheduling
Date Mon, 04 May 2015 03:42:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-3547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526210#comment-14526210

Xianyin Xin commented on YARN-3547:

Thanks [~leftnoteasy]! 
Attached a new patch using Set. When i review the patch, I found patch-001 is ill-considered,
because unlike CS, Fair uses list to organize the apps at the very beginning, thus the newly
added sorted set may become un-sorted because of the "update resource usage" or "update fair
share" operations. We have two choices, the first is we continue to use the solution in patch-001,
but fix it with considerations that we reorder the set when we update resource usage or fair
shares just like in CS; the second is like patch-003, before we do assign, we scan the runningApps,
and pick out the apps with real request to construct a set. Patch-003 is more efficient than
patch-002, and the change of code is less than patch-001 (if fixed). What do you think, [~leftnoteasy]?
Change the name "pendingApps" to "pendingForResourceApps" because "pending" is ambiguous.
Maybe we need a better name.

> FairScheduler: Apps that have no resource demand should not participate scheduling
> ----------------------------------------------------------------------------------
>                 Key: YARN-3547
>                 URL: https://issues.apache.org/jira/browse/YARN-3547
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: fairscheduler
>            Reporter: Xianyin Xin
>            Assignee: Xianyin Xin
>         Attachments: YARN-3547.001.patch, YARN-3547.002.patch
> At present, all of the 'running' apps participate the scheduling process, however, most
of them may have no resource demand on a production cluster, as the app's status is running
other than waiting for resource at the most of the app's lifetime. It's not a wise way we
sort all the 'running' apps and try to fulfill them, especially on a large-scale cluster which
has heavy scheduling load. 

This message was sent by Atlassian JIRA

View raw message