hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mathias Herberts (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-3310) Custom grouping comparator cannot be set for Combiners
Date Sat, 29 Oct 2011 17:23:32 GMT
Custom grouping comparator cannot be set for Combiners

                 Key: MAPREDUCE-3310
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3310
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: client
    Affects Versions: 0.21.0,,, 0.20.2, 0.20.1
         Environment: All
            Reporter: Mathias Herberts

Combiners are often described as 'Reducers running on the Map side'.

As Reducers, Combiners are fed <K,{V}>, where {V} is built by grouping values associated
with the 'same' key.

For Reducers, the comparator used for grouping values can be set independently of that used
to sort the keys (using Job.setGroupingComparatorClass).

Such a configuration is not possible for Combiners, meaning some things done in Reducers cannot
be done in Combiners (such as secondary sort).

It would be handy to have a Job.setCombinerGroupingComparatorClass method that would allow
the setting of the grouping comparator used when applying a Combiner.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message