hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "W.P. McNeill" <bill...@gmail.com>
Subject How do I serialize a MapWritable value?
Date Mon, 18 Apr 2011 21:11:34 GMT
I have a reducer that outputs pairs of the form <Text, MapWritable>.  The
MapWritable is a map from Text to Long.  The key and output values of my
reducer are declared like so:

      job.setOutputKeyClass(Text.class);
      job.setOutputValueClass(MapWritable.class);

The last few lines of my reducer look like this:

         MapWritable value = new MapWritable();
         for (Map.Entry<String, Long> entry : totalWordCount.entrySet())
            value.put(new Text(entry.getKey()), new
LongWritable(entry.getValue()));
         context.write(key, value);

In the debugger I can see that my MapWritable value is a map with the values
I expect it to contain.  However, when I run the MapReduce process the
output literally looks like this:

key1      org.apache.hadoop.io.MapWritable@396cbd97
key2         org.apache.hadoop.io.MapWritable@17991de1
key3         org.apache.hadoop.io.MapWritable@18f63055

The values are descriptive stringifications of the MapWritable class without
any data.  What I want instead are serializations of the maps that contain
their data. I thought the point of the MapWritable class was that it handled
serialization for you. Is there a step I'm missing?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message