hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Ferguson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-326) Add multi-resource scheduling to the fair scheduler
Date Mon, 29 Apr 2013 02:24:16 GMT

    [ https://issues.apache.org/jira/browse/YARN-326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13644208#comment-13644208
] 

Andrew Ferguson commented on YARN-326:
--------------------------------------

[~sandyr] bingo. that was exactly the concern I alluded to before. glad we found it while
thinking about the design. :-)

[~kkambatl] yup, that's the idea -- fractional min-share, which would be interpreted as a
fraction of the dominant resource (which wouldn't be pre-specified, so the queue's dominant
resource could adapt based on the jobs submitted) ... I wrote my example a bit quickly, sorry!
let me know if something's still not clear.

the new plan sounds like a good approach. I like it.


                
> Add multi-resource scheduling to the fair scheduler
> ---------------------------------------------------
>
>                 Key: YARN-326
>                 URL: https://issues.apache.org/jira/browse/YARN-326
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: scheduler
>    Affects Versions: 2.0.2-alpha
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>         Attachments: FairSchedulerDRFDesignDoc.pdf, YARN-326.patch, YARN-326.patch
>
>
> With YARN-2 in, the capacity scheduler has the ability to schedule based on multiple
resources, using dominant resource fairness.  The fair scheduler should be able to do multiple
resource scheduling as well, also using dominant resource fairness.
> More details to come on how the corner cases with fair scheduler configs such as min
and max resources will be handled.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message