Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Tue, 15 May 2012 21:21:09 +0000 (UTC)
From: "Matei Zaharia (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: 
 <332854514.1293.1337116869219.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <63160998.1186.1321934680645.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (MAPREDUCE-3451) Port Fair Scheduler to MR2
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276225#comment-13276225 ] 

Matei Zaharia commented on MAPREDUCE-3451:
------------------------------------------

Having a single very feature-rich scheduler is certainly useful, but that's a separate discussion. My point was just that some people want compatibility with the MR1 Fair Scheduler as a feature -- not having that is an obstacle to their adopting MR2.

I also think it would be hard to design a single scheduler that will support all the features people might want in the future, for the same reason that it was hard to get different development groups to build a single scheduler in MR1. You just can't anticipate all the needs, and how various features will interact -- for example, Yahoo! had to remove preemption from the capacity scheduler because it complicated the design. We are also planning to add some features to the Fair Scheduler ourselves, such as multi-resource fairness, that might require major changes to other schedulers even if we were to wait on them adopting the existing features.
                
> Port Fair Scheduler to MR2
> --------------------------
>
>                 Key: MAPREDUCE-3451
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3451
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: mrv2, scheduler
>            Reporter: Patrick Wendell
>            Assignee: Patrick Wendell
>         Attachments: MAPREDUCE-3451.v1.patch.txt, MAPREDUCE-3451.v2.patch.txt, MAPREDUCE-3451.v3.patch.txt, MAPREDUCE-3451.v4.patch.txt, MAPREDUCE-3451.v5.patch
>
>
> The Fair Scheduler is in widespread use today in MR1 clusters, but not yet ported to MR2. This is to track the porting of the Fair Scheduler to MR2 and will be updated to include design considerations and progress.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira