hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stanley Xu <wenhao...@gmail.com>
Subject Could we use different output Format for the Mapper and Combiner?
Date Wed, 16 Feb 2011 11:02:04 GMT
Dear all,

I am writing a map-reduce job today. Which I hope I could use different
format for the Mapper and Combiner. I am using the Text as the format of the
Mapper and MapWritable as the format of the format.

But it looks the hadoop didn't support that yet?

I have some code like the following:

public class RawLogMapper extends Mapper<LongWritable, Text, Text, Text> {

public class RawLogCombiner extends Reducer<Text, Text, Text, MapWritable> {

job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(Text.class);

job.setOutputKeyClass(Text.class);
job.setOutputValueClass(MapWritable.class);
job.setOutputFormatClass(TextOutputFormat.class);

But it failed and the logs told me that there are type mismatch. Is there
anyway I could use different type for the VALUEOUT for the mapper and
combiner?

Thanks


Best wishes,
Xu Wenhao

Mime
View raw message