Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C0FF87C37 for ; Mon, 18 Jul 2011 21:22:43 +0000 (UTC) Received: (qmail 17702 invoked by uid 500); 18 Jul 2011 21:22:42 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 17649 invoked by uid 500); 18 Jul 2011 21:22:42 -0000 Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-user@hadoop.apache.org Delivered-To: mailing list mapreduce-user@hadoop.apache.org Received: (qmail 17641 invoked by uid 99); 18 Jul 2011 21:22:41 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 18 Jul 2011 21:22:41 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of joey@cloudera.com designates 209.85.161.51 as permitted sender) Received: from [209.85.161.51] (HELO mail-fx0-f51.google.com) (209.85.161.51) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 18 Jul 2011 21:22:35 +0000 Received: by fxh10 with SMTP id 10so7417468fxh.38 for ; Mon, 18 Jul 2011 14:22:14 -0700 (PDT) MIME-Version: 1.0 Received: by 10.223.120.134 with SMTP id d6mr4875835far.112.1311024133663; Mon, 18 Jul 2011 14:22:13 -0700 (PDT) Received: by 10.223.70.199 with HTTP; Mon, 18 Jul 2011 14:22:13 -0700 (PDT) In-Reply-To: References: Date: Mon, 18 Jul 2011 17:22:13 -0400 Message-ID: Subject: Re: ArrayWritable Doesn't Write From: Joey Echeverria To: mapreduce-user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001636ef0a61c76a9204a85e979d --001636ef0a61c76a9204a85e979d Content-Type: text/plain; charset=ISO-8859-1 ArrayWritables can't be deserialized because they don't encode the type of the objects with the data. The solution is to sub-class ArrayWritable with your specific type. In your case, you'd need to do this: public class IntArraryWritable { public IntArrayWritable() { super(IntWritable.class); } public IntArrayWritable(IntWritable[] data) { super(IntWritable.class, data); } } Then use IntArrayWritable in your code segments below. You can see this documented in the java docs: http://hadoop.apache.org/common/docs/r0.20.203.0/api/org/apache/hadoop/io/ArrayWritable.html -Joey On Mon, Jul 18, 2011 at 3:57 PM, Geoffry Roberts wrote: > All, > > For the first time I have tried to use the class ArrayWritable. All goes > well enough until the Reducer tries to do a write. Then, I get the > following exception: > > java.lang.RuntimeException: java.lang.NoSuchMethodException: > org.apache.hadoop.io.ArrayWritable.() > ... > > From the reducer, the relevant code. > > // Prepare the output. > IntWritable[] out = new IntWritable[5]; > // populate output array. > ctx.write(key, new ArrayWritable(IntWritable.class, out)); > // If I comment the above line out, the job runs without issue. > > Can anyone see what I'm doing wrong? > -- > Geoffry Roberts > > -- Joseph Echeverria Cloudera, Inc. 443.305.9434 --001636ef0a61c76a9204a85e979d Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable ArrayWritables can't be deserialized because they don't encode the = type of the objects with the data. The solution is to sub-class ArrayWritab= le with your specific type. In your case, you'd need to do this:

public class IntArraryWritable {

= =A0 public IntArrayWritable() {
=A0 =A0 super(IntWritable.class);=
=A0 }

=A0 public IntArrayWritable(IntWr= itable[] data) {
=A0 =A0 super(IntWritable.class, data);
=A0 }
}

Then use IntArrayWritable in your code segments belo= w. You can see this documented in the java docs:

<= meta http-equiv=3D"content-type" content=3D"text/html; charset=3Dutf-8">http://hadoop.apache.org/common/docs/r0.20.203.0= /api/org/apache/hadoop/io/ArrayWritable.html

-Joey

On Mon, Jul 18,= 2011 at 3:57 PM, Geoffry Roberts <geoffry.roberts@gmail.com> wrote:
All,

For the first time I have tried= to use the class ArrayWritable.=A0 All goes well enough until the Reducer = tries to do a write.=A0 Then, I get the following exception:

java.la= ng.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.io.= ArrayWritable.<init>()
...

From the reducer, the relevant code.

// Prepare the outp= ut.
IntWritable[] out = =3D new IntWritable[5];
// populate output array.
ctx.write(key, new ArrayWri= table(IntWritable.class, out));
// If I comment the above line ou= t, the job runs without issue.

Can anyone see what I'm doing wrong?
--
Geoffry Roberts




--
Joseph Eche= verria
Cloudera, Inc.
443.305.9434

--001636ef0a61c76a9204a85e979d--