hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "B Anil Kumar (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3962) InverseMapper causes IntWritable type to collected as Text
Date Thu, 08 Mar 2012 06:43:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225039#comment-13225039
] 

B Anil Kumar commented on MAPREDUCE-3962:
-----------------------------------------

Hi Saad,

If you don't specify FileInputFormat explicitly, then It uses TextInputFormat as defalut.
In this case, map input key will be LongWritable and Value will be Text.

And also you need to set MapOutputKeyClass, MapOutputValueClass, OutPutKeyClass and OutPutValueClass.

Please take care of above ponits and run the job. 

If you can run the job successfully, then please close this issue as invalid.

                
> InverseMapper causes IntWritable type to collected as Text
> ----------------------------------------------------------
>
>                 Key: MAPREDUCE-3962
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3962
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 1.0.0
>            Reporter: Saad Patel
>
> When using the org.apache.hadoop.mapred.lib.InverseMapper, key,values are inversed as
expected. For example (Text,IntWritable) will get inversed. However, output key,value only
works if you use (Text, Text). Below is an example, where I was chaining 2 jobs:
> Job 1:
> public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable,
Text, IntWritable>{...}
> Job 2:
> conf.setMapperClass(InverseMapper.class);
> public static class Reduce extends MapReduceBase implements Reducer<IntWritable, Text,
Text, IntWritable> {...} //I would expect this to work. When I do this, I get
> java.lang.ClassCastException: org.apache.hadoop.io.Text cannot be cast to org.apache.hadoop.io.IntWritable
> To re-inverse my key,values, I had to do this:
> public static class Reduce extends MapReduceBase implements Reducer<Text, Text, Text,
Text>
> Notice that in order for the reducer to properly accept key,values, I had to indicate
the key as Text.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message