hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mariappan Asokan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-2454) Allow external sorter plugin for MR
Date Mon, 12 Nov 2012 18:45:16 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mariappan Asokan updated MAPREDUCE-2454:
----------------------------------------

    Attachment: mapreduce-2454.patch

Hi Alejandro,
  I made minor changes in annotations.  Also, I am summarizing the visibility changes in the
following table(Hope it is formatted correctly.  If not I will attach a separate file to this
Jira):

||Class||Protection||Annotation||Reason||
|{{org.apache.hadoop.mapred.MapTask}}|Changed from package protected to public|Annotated as
Public, Unstable|To be able to access inner interface {{PostMapProcessor}} and inner class
{{MapOutputBuffer}} from external plug-ins.|
|{{org.apache.hadoop.mapred.MapTask.MapOutputCollector}}|Name changed to PostMapProcessor
and protection changed from package protected to public|Annotated as Public, Unstable|To be
able to implement external plug-ins.|
|{{org.apache.hadoop.mapred.MapTask.MapOutputBuffer}}|Changed to a static inner class and
protection changed from package protected to public|Annotated as Public, Unstable|To be able
to use the default implementation of {{PostMapProcessor}}.|
|{{org.apache.hadoop.mapred.SpillRecord}}|Changed from package protected to public|Annotated
as Public, Unstable|To be able to write the intermediate output from {{PostMapProcessor}}
plug-ins.|
|{{org.apache.hadoop.mapred.Task.CombinerRunner}}|Changed from protected to public|Changed
from Private, Unstable to Public, Unstable|For code reuse to run the {{Combiner}} from external
{{PostMapProcessor}} plug-ins.|
|{{org.apache.hadoop.mapred.Task.TaskReporter}}|Changed from protected to public|Changed from
Private, Unstable to Public, Unstable|Has to be passed to {{CombinerRunner}} from external
{{PostMapProcessor}} plug-ins.|
|{{org.apache.hadoop.mapreduce.task.reduce.ExceptionReporter}}|Changed from package protected
to public|Annotated as Public, Unstable|To report exception to {{Shuffle}} from {{PreReduceProcessor.}}|
|{{org.apache.hadoop.mapreduce.task.reduce.MapHost}}|Changed from package protected to public|Annotated
as Public, Unstable|Passed to {{shuffle()}} method in {{MapOutput.}}|
|{{org.apache.hadoop.mapreduce.task.reduce.MapOutput}}|Changed from package protected to public|Annotated
as Public, Unstable|Returned from {{reserve()}} method in {{PreReduceProcessor.}}|
|{{org.apache.hadoop.mapreduce.task.reduce.MergeManager}}|No change|Changed from Private,
Unstable to Public, Unstable|To reuse the default implementation of {{PreReduceProcessor.}}|
|{{org.apache.hadoop.mapreduce.task.reduce.Shuffle}}|No change|Changed from Private, Unstable
to Public, Unstable|Passed to external {{PreReduceProcessor}} plug-ins.|
|{{org.apache.hadoop.mapreduce.task.reduce.ShuffleClientMetrics}}|Changed from package protected
to public|Annotated as Public, Unstable|Passed to {{shuffle()}} method in external {{MapOutput.}}|


                
> Allow external sorter plugin for MR
> -----------------------------------
>
>                 Key: MAPREDUCE-2454
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2454
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>    Affects Versions: 2.0.0-alpha, 3.0.0, 2.0.2-alpha
>            Reporter: Mariappan Asokan
>            Assignee: Mariappan Asokan
>            Priority: Minor
>              Labels: features, performance, plugin, sort
>         Attachments: HadoopSortPlugin.pdf, HadoopSortPlugin.pdf, KeyValueIterator.java,
MapOutputSorterAbstract.java, MapOutputSorter.java, mapreduce-2454.patch, mapreduce-2454.patch,
mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch,
mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch,
mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch,
mapreduce-2454.patch, mr-2454-on-mr-279-build82.patch.gz, MR-2454-trunkPatchPreview.gz, ReduceInputSorter.java
>
>
> Define interfaces and some abstract classes in the Hadoop framework to facilitate external
sorter plugins both on the Map and Reduce sides.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message