hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matei Zaharia (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-279) Map-Reduce 2.0
Date Fri, 18 Feb 2011 01:31:13 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12996189#comment-12996189

Matei Zaharia commented on MAPREDUCE-279:

+1 on decoupling Hadoop MapReduce from the resource management system in a way that allows
Hadoop to run on top of other cluster scheduling systems as well. Apart from simplifying experimentation
with these types of two-level schedulers, I think this is would be a good thing for the MapReduce
project in general as a way to make the project runnable in the maximum variety of environments.
For example, there have already been efforts to get Hadoop running on HPC schedulers (e.g.
Grid Engine) or Condor, and that would be quite a bit easier with the refactoring that Arun
is doing. I imagine that there will be a lot of other work in cluster scheduling in future
years, especially as people start running more non-MapReduce applications, so it would be
nice to be able to run the Hadoop software stack in these environments.

> Map-Reduce 2.0
> --------------
>                 Key: MAPREDUCE-279
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-279
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobtracker, tasktracker
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>             Fix For: 0.23.0
> Re-factor MapReduce into a generic resource scheduler and a per-job, user-defined component
that manages the application execution. 

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message