hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Olga Natkovich (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-3586) keep combiner backward compatible with earlier versions of hadoop
Date Tue, 17 Jun 2008 22:28:44 GMT
keep combiner backward compatible with earlier versions of hadoop
-----------------------------------------------------------------

                 Key: HADOOP-3586
                 URL: https://issues.apache.org/jira/browse/HADOOP-3586
             Project: Hadoop Core
          Issue Type: Bug
            Reporter: Olga Natkovich
             Fix For: 0.18.0


In hadoop 16 and earlier, the combiner was guaranteed to run once and only once for each map.
In 17 this compatibility was slightly broken: the combiner does not run if a single <K,V>
occupies the entire sort buffer. In 18, this is further changed to where the combiner can
be called multiple times on both map and reduce sides.

This breaks Pig's current implementation of the combiner and it is not easy to fix in a short
period of time.

We would like to ask that for a way for an application to ask for a backward compatible behavior
for some period of time until it can adjust to the new behavior.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message