hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrzej Bialecki (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-920) MapFileOutputFormat and SequenceFileOutputFormat use incorrect key/value classes in map/reduce tasks
Date Tue, 23 Jan 2007 16:19:49 GMT
MapFileOutputFormat and SequenceFileOutputFormat use incorrect key/value classes in map/reduce
tasks
----------------------------------------------------------------------------------------------------

                 Key: HADOOP-920
                 URL: https://issues.apache.org/jira/browse/HADOOP-920
             Project: Hadoop
          Issue Type: Bug
          Components: mapred
    Affects Versions: 0.11.0
            Reporter: Andrzej Bialecki 
             Fix For: 0.11.0


Let's assume a job uses different key/value class for the output of map tasks and for the
final output of reduce tasks.

When executing map tasks classes returned from JobConf.getMapOutputKeyClass() / getMapOutputValueClass()
should be used, and when executing reduce tasks classes returned from JobConf.gtOutputKeyClass()
/ getOutputValueClass() should be used.

Currently both map and reduce tasks will use getMapOutputKeyClass/getMapOutputValueClass when
using MapFileOutputFormat, or they will always use getOutputKeyClassgetOutputValueClass when
using SequenceFileOutputFormat. This causes exceptions, because Mapper / Reducer implementations
will output different key/value classes than expected.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message