hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joey Echeverria <j...@cloudera.com>
Subject Re: ArrayWritable Doesn't Write
Date Mon, 18 Jul 2011 21:22:13 GMT
ArrayWritables can't be deserialized because they don't encode the type of
the objects with the data. The solution is to sub-class ArrayWritable with
your specific type. In your case, you'd need to do this:

public class IntArraryWritable {

  public IntArrayWritable() {
    super(IntWritable.class);
  }

  public IntArrayWritable(IntWritable[] data) {
    super(IntWritable.class, data);
  }
}

Then use IntArrayWritable in your code segments below. You can see this
documented in the java docs:

http://hadoop.apache.org/common/docs/r0.20.203.0/api/org/apache/hadoop/io/ArrayWritable.html

-Joey

On Mon, Jul 18, 2011 at 3:57 PM, Geoffry Roberts
<geoffry.roberts@gmail.com>wrote:

> All,
>
> For the first time I have tried to use the class ArrayWritable.  All goes
> well enough until the Reducer tries to do a write.  Then, I get the
> following exception:
>
> java.lang.RuntimeException: java.lang.NoSuchMethodException:
> org.apache.hadoop.io.ArrayWritable.<init>()
> ...
>
> From the reducer, the relevant code.
>
> // Prepare the output.
> IntWritable[] out = new IntWritable[5];
> // populate output array.
> ctx.write(key, new ArrayWritable(IntWritable.class, out));
> // If I comment the above line out, the job runs without issue.
>
> Can anyone see what I'm doing wrong?
> --
> Geoffry Roberts
>
>


-- 
Joseph Echeverria
Cloudera, Inc.
443.305.9434

Mime
View raw message