hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-920) MapFileOutputFormat and SequenceFileOutputFormat use incorrect key/value classes in map/reduce tasks
Date Tue, 23 Jan 2007 17:23:49 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12466773
] 

Hadoop QA commented on HADOOP-920:
----------------------------------

+1, because http://issues.apache.org/jira/secure/attachment/12349457/key-value-class.patch
applied and successfully tested against trunk revision r498829.

> MapFileOutputFormat and SequenceFileOutputFormat use incorrect key/value classes in map/reduce
tasks
> ----------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-920
>                 URL: https://issues.apache.org/jira/browse/HADOOP-920
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.11.0
>            Reporter: Andrzej Bialecki 
>             Fix For: 0.11.0
>
>         Attachments: key-value-class.patch
>
>
> Let's assume a job uses different key/value class for the output of map tasks and for
the final output of reduce tasks.
> When executing map tasks classes returned from JobConf.getMapOutputKeyClass() / getMapOutputValueClass()
should be used, and when executing reduce tasks classes returned from JobConf.gtOutputKeyClass()
/ getOutputValueClass() should be used.
> Currently both map and reduce tasks will use getMapOutputKeyClass/getMapOutputValueClass
when using MapFileOutputFormat, or they will always use getOutputKeyClassgetOutputValueClass
when using SequenceFileOutputFormat. This causes exceptions, because Mapper / Reducer implementations
will output different key/value classes than expected.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message