hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-279) Map-Reduce 2.0
Date Thu, 26 May 2011 23:35:50 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13039993#comment-13039993
] 

Tom White commented on MAPREDUCE-279:
-------------------------------------

I'm wondering what the maven modules might look like for this when integrated into trunk.
Something like:

* api - containing the user-facing public API of MapReduce (from org.apache.hadoop.mapred(uce)).
When MAPREDUCE-1638 is done it will be possible to split the API into a self-contained tree
(no dependencies on other parts of MapReduce). 
* lib - containing the user-facing public MapReduce libraries (from org.apache.hadoop.mapred
and org.apache.hadoop.mapred(uce).lib). There's a patch in MAPREDUCE-1478 to perform this
separation.
* classic-impl - containing the implementation classes for MapReduce. This is what's left
over after doing MAPREDUCE-1638 and MAPREDUCE-1478.
* nextgen-impl - this is mr-client in the MR-279 branch, which I think should be renamed,
since it's not immediately clear what it's a client of in the context of the whole MapReduce
project. It has submodules app, common, hs, jobclient, shuffle.
* yarn - the yarn framework from the MR-279 branch. Yarn is broken into submodules too.

Given the progress on mavenizing common (HADOOP-6671), is it worth integrating MAPREDUCE-279
at the same time as doing the full Mavenization of MapReduce? That would seem ideal, but perhaps
there's an alternative I haven't considered. 


> Map-Reduce 2.0
> --------------
>
>                 Key: MAPREDUCE-279
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-279
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobtracker, tasktracker
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>             Fix For: 0.23.0
>
>         Attachments: MR-279.patch, MR-279.patch, MR-279.sh, MR-279_MR_files_to_move.txt,
capacity-scheduler-dark-theme.png, multi-column-stable-sort-default-theme.png, yarn-state-machine.job.dot,
yarn-state-machine.job.png, yarn-state-machine.task-attempt.dot, yarn-state-machine.task-attempt.png,
yarn-state-machine.task.dot, yarn-state-machine.task.png
>
>
> Re-factor MapReduce into a generic resource scheduler and a per-job, user-defined component
that manages the application execution. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message