hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3702) add support for chaining Maps in a single Map and after a Reduce [M*/RM*]
Date Wed, 03 Sep 2008 15:23:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12628024#action_12628024
] 

Enis Soztutar commented on HADOOP-3702:
---------------------------------------

bq. Still, I don't agree on making the Chain* classes generic. Their generics would not be
use for any type checking at compilation. And it is not possible to get rid of the @SuppressWarnings("unchecked")
.
I do not wish to further delay the patch for this. I'm OK with the current patch, although
the reasons for using generics remain.

bq. Shouldn't be appropriate to make them?...
Sigh. yes, since in generics, G<Foo> is not a subtype of G<Bar> even if Foo extends
Bar. So please make that change. 



> add support for chaining Maps in a single Map and after a Reduce [M*/RM*]
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-3702
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3702
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: mapred
>         Environment: all
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>             Fix For: 0.19.0
>
>         Attachments: Hadoop-3702.patch, patch3702.txt, patch3702.txt, patch3702.txt,
patch3702.txt, patch3702.txt, patch3702.txt, patch3702.txt, patch3702.txt, patch3702.txt,
patch3702.txt, patch3702.txt, patch3702.txt
>
>
> On the same input, we usually need to run multiple Maps one after the other without no
Reduce. We also have to run multiple Maps after the Reduce.
> If all pre-Reduce Maps are chained together and run as a single Map a significant amount
of Disk I/O will be avoided. 
> Similarly all post-Reduce Maps can be chained together and run in the Reduce phase after
the Reduce.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message