hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stuart Sierra" <m...@stuartsierra.com>
Subject LongSumReducer and TextOutputFormat
Date Tue, 29 Jul 2008 15:55:56 GMT
Hello all,

I tried using o.a.h.mapred.lib.LongSumReducer in a wordcount-like job.
 My custom Mapper outputs Text keys and LongWritable values.  I
expected to get the Text keys in the final output.  But instead, I got
this:

org.apache.hadoop.mapred.MapTask$MapOutputBuffer@1326484     12286
org.apache.hadoop.mapred.MapTask$MapOutputBuffer@16546ef     5037
org.apache.hadoop.mapred.MapTask$MapOutputBuffer@16925b0     29786
org.apache.hadoop.mapred.MapTask$MapOutputBuffer@16c9867     1219
org.apache.hadoop.mapred.MapTask$MapOutputBuffer@16d2633     1335
org.apache.hadoop.mapred.MapTask$MapOutputBuffer@1742700     1705
org.apache.hadoop.mapred.MapTask$MapOutputBuffer@17f409c     3178

My job configuration looks like this:

        job.setMapperClass(MyMap.class);
        job.setReducerClass(LongSumReducer.class);

        job.setInputFormat(SequenceFileInputFormat.class);

        job.setOutputFormat(TextOutputFormat.class);
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(LongWritable.class);

Anybody know what's going on here?  This is Hadoop 0.17.1.

Thanks,
-Stuart

Mime
View raw message