hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-279) Map-Reduce 2.0
Date Fri, 18 Mar 2011 04:44:32 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008327#comment-13008327
] 

Tom White commented on MAPREDUCE-279:
-------------------------------------

There's a lot to digest here, but here are a couple of quick initial high-level comments from
a packaging and staging perspective.

I wonder if it would be easier not to move the src/java/org/apache/hadoop/mapred(uce) trees
at this stage. MR 2 could just depend on the MapReduce JAR produced by the ant file, just
like it does for Common. This would make the introduction of the codebase easier. There are
some changes required in the existing classes, but by the look of things they are fairly minor
and by introducing them in situ (in separate JIRAs) we can be sure they won't break existing
users, and the changes would be easier to track.

Alternatively this work could depend on full mavenization (at least of MapReduce), but that's
probably some way off.

MAPREDUCE-1638 is highly relevant for this work, since it aims to split out the MR API from
the implementation. I've got an in-progress patch for this, which I'll post soon for discussion.



> Map-Reduce 2.0
> --------------
>
>                 Key: MAPREDUCE-279
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-279
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobtracker, tasktracker
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>             Fix For: 0.23.0
>
>         Attachments: MR-279.patch, MR-279.patch, MR-279.sh, MR-279_MR_files_to_move.txt
>
>
> Re-factor MapReduce into a generic resource scheduler and a per-job, user-defined component
that manages the application execution. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message