hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bob Cummins (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-2369) Using TableMapper Iterable IntWritables not passed to the reducer in order put by mapper
Date Tue, 08 Mar 2011 19:42:59 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13004157#comment-13004157
] 

Bob Cummins commented on MAPREDUCE-2369:
----------------------------------------

Tom,

  Thanks for your attention.

 My issue was with the order or multiple values written to one specific key, which could be
controlled by the mapper. The reducer could ingest the key and more than one value, and if
the values were in a predictable order, each value associated with a particular key could
be identified under a "contract" between the mapper and the reducer. 

The issue of multiple mappers funneling key/pair(s) sets to a reducer would be irrelevant
to this case.

I am sorry if this was not clear.

Bob



Robert T. Cummins, Jr.    CEH, LPIC-1, CREA, CPT, GSEC, Network+


> Using TableMapper Iterable IntWritables not passed to the reducer in order put by mapper
> ----------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2369
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2369
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: client
>    Affects Versions: 0.20.2
>         Environment: Cloudera VM 3.5
>            Reporter: Bob Cummins
>            Priority: Minor
>
> For mapper class:
>       class Mapper1 extends TableMapper<ImmutableBytesWritable,IntWritable>
> With reducer class:
>      class Reducer1 extends TableReducer<ImmutableBytesWritable,IntWritable, ImmutableBytesWritable>
> Iterable<IntWritable> values are usually received by the reducer in the
> order the values are written to the context by the mapper. However in my
> testing about 5% of cases, the same order is not maintained, and the ability
> of the reducer to categorize a value by order lost.
> Chronological order guaranteed would serve as a facility for identification by the reducer.
>  
>  

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message