hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matei Zaharia (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3451) Port Fair Scheduler to MR2
Date Tue, 15 May 2012 21:21:09 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276225#comment-13276225
] 

Matei Zaharia commented on MAPREDUCE-3451:
------------------------------------------

Having a single very feature-rich scheduler is certainly useful, but that's a separate discussion.
My point was just that some people want compatibility with the MR1 Fair Scheduler as a feature
-- not having that is an obstacle to their adopting MR2.

I also think it would be hard to design a single scheduler that will support all the features
people might want in the future, for the same reason that it was hard to get different development
groups to build a single scheduler in MR1. You just can't anticipate all the needs, and how
various features will interact -- for example, Yahoo! had to remove preemption from the capacity
scheduler because it complicated the design. We are also planning to add some features to
the Fair Scheduler ourselves, such as multi-resource fairness, that might require major changes
to other schedulers even if we were to wait on them adopting the existing features.
                
> Port Fair Scheduler to MR2
> --------------------------
>
>                 Key: MAPREDUCE-3451
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3451
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: mrv2, scheduler
>            Reporter: Patrick Wendell
>            Assignee: Patrick Wendell
>         Attachments: MAPREDUCE-3451.v1.patch.txt, MAPREDUCE-3451.v2.patch.txt, MAPREDUCE-3451.v3.patch.txt,
MAPREDUCE-3451.v4.patch.txt, MAPREDUCE-3451.v5.patch
>
>
> The Fair Scheduler is in widespread use today in MR1 clusters, but not yet ported to
MR2. This is to track the porting of the Fair Scheduler to MR2 and will be updated to include
design considerations and progress.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message