hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Radim Kolar (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-5153) Support for running combiners without reducers
Date Tue, 16 Apr 2013 15:41:15 GMT
Radim Kolar created MAPREDUCE-5153:
--------------------------------------

             Summary: Support for running combiners without reducers
                 Key: MAPREDUCE-5153
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5153
             Project: Hadoop Map/Reduce
          Issue Type: New Feature
            Reporter: Radim Kolar


scenario: Workflow mapper -> sort -> combiner -> hdfs

No api change is need, if user set combiner class and reducers = 0 then run combiner and sent
output to HDFS.

Popular libraries such as scalding and cascading are offering this functionality, but they
use caching entire mapper output in memory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message