hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2454) Allow external sorter plugin for MR
Date Mon, 19 Nov 2012 05:55:11 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500031#comment-13500031
] 

Alejandro Abdelnur commented on MAPREDUCE-2454:
-----------------------------------------------

Arun, 

Thanks for taking the time to help me review this JIRA, I really appreciate that. I also agree
this JIRA is in the core of MR and merits for extra attention it got.

The current approach, which has been done following your recommendations, has been already
thoroughly reviewed by you and myself. Plus the current size of the patch is approx 100K,
with 33K of it in testcases, less than 15K of new code and the rest a straight forward refactoring
of existing code. 

All the feedback you've provided has been integrated in the patch. 

There is only one discussion left, that you brought up, regarding 'passing a shuffle down
to the merger'.

Because of this, I don't think it is necessary to break up this JIRA in sub JIRAs at this
point.

Once we decide on this pending issue I want to commit this patch. Any further improvement/change/fix
can be done incrementally with follow up JIRAs in the same way we work in the rest of Hadoop.


Cheers

                
> Allow external sorter plugin for MR
> -----------------------------------
>
>                 Key: MAPREDUCE-2454
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2454
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>    Affects Versions: 2.0.0-alpha, 3.0.0, 2.0.2-alpha
>            Reporter: Mariappan Asokan
>            Assignee: Mariappan Asokan
>            Priority: Minor
>              Labels: features, performance, plugin, sort
>         Attachments: HadoopSortPlugin.pdf, HadoopSortPlugin.pdf, KeyValueIterator.java,
MapOutputSorterAbstract.java, MapOutputSorter.java, mapreduce-2454-modified-code.patch, mapreduce-2454-modified-test.patch,
mapreduce-2454-new-test.patch, mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch,
mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch,
mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch,
mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch,
mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454-protection-change.patch,
mr-2454-on-mr-279-build82.patch.gz, MR-2454-trunkPatchPreview.gz, ReduceInputSorter.java
>
>
> Define interfaces and some abstract classes in the Hadoop framework to facilitate external
sorter plugins both on the Map and Reduce sides.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message