hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Radim Kolar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5153) Support for running combiners without reducers
Date Fri, 26 Jul 2013 12:35:50 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13720740#comment-13720740

Radim Kolar commented on MAPREDUCE-5153:

its very simple to implement. 

If you want to push things forward then do it.
> Support for running combiners without reducers
> ----------------------------------------------
>                 Key: MAPREDUCE-5153
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5153
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>            Reporter: Radim Kolar
> scenario: Workflow mapper -> sort -> combiner -> hdfs
> No api change is need, if user set combiner class and reducers = 0 then run combiner
and sent output to HDFS.
> Popular libraries such as scalding and cascading are offering this functionality, but
they use caching entire mapper output in memory.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message