Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 10B8FE3BD for ; Wed, 27 Feb 2013 19:50:59 +0000 (UTC) Received: (qmail 49874 invoked by uid 500); 27 Feb 2013 19:50:54 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 49689 invoked by uid 500); 27 Feb 2013 19:50:54 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 49678 invoked by uid 99); 27 Feb 2013 19:50:54 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Feb 2013 19:50:54 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of sandy.ryza@cloudera.com designates 74.125.83.48 as permitted sender) Received: from [74.125.83.48] (HELO mail-ee0-f48.google.com) (74.125.83.48) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Feb 2013 19:50:49 +0000 Received: by mail-ee0-f48.google.com with SMTP id t10so856605eei.35 for ; Wed, 27 Feb 2013 11:50:28 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type:x-gm-message-state; bh=TJQNEF4uCC3PG/ZnAqYa8kT2/+xJ0h0438KYOTpp6rI=; b=hqHFJe5W2GlN16CRGWmooXK3S/43yRlOn/wNOScpQRhIYbddp9VcuxGL2fkGZehdjZ KZzK7h2tk1sN5YQ4mI4wamrlCKdavqzeV5yK3gqejfF0PntdUGTRqCbD6gINzWYjO2An jI9LtiovdztWR9CpObSXj3ql2BT/kYjB3seb3B/0rzrvok385kKhPUFy5p16epUopBZU NIwiduwuF0Dp2yNf56t4CaC89p+PQ5lqosJQJ8k2jlIzkYURQgvotioguOG2iN68QJ/c C9qWsLd/A6i2w+IACgH4FL6yAT1qrBMlxrJc1mLsHzMV13EV1MYWXLpOl72C8oblOscO LiNg== MIME-Version: 1.0 X-Received: by 10.14.215.193 with SMTP id e41mr9092365eep.32.1361994627988; Wed, 27 Feb 2013 11:50:27 -0800 (PST) Received: by 10.14.101.135 with HTTP; Wed, 27 Feb 2013 11:50:27 -0800 (PST) In-Reply-To: References: Date: Wed, 27 Feb 2013 11:50:27 -0800 Message-ID: Subject: Re: Custom output value for map function From: Sandy Ryza To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=e89a8f647ac3fdd7bf04d6ba151a X-Gm-Message-State: ALoCoQkGWKvePb3NNuxVRpSxvFD3VvqpwhxpVpoYBV5+ZXE2qr0otOUsV3OsXreDFMJOczFQY55u X-Virus-Checked: Checked by ClamAV on apache.org --e89a8f647ac3fdd7bf04d6ba151a Content-Type: text/plain; charset=ISO-8859-1 That's right, the date needs to be written and read in the same order. On Wed, Feb 27, 2013 at 11:04 AM, Paul van Hoven < paul.van.hoven@googlemail.com> wrote: > Great! Thank you. > > I guess the order for writing and reading the data this way is > important. I mean, for > > out.writeUTF("blabla") > out.writeInt(12) > > the following would be correct > > text = in.readUTF(); > number = in.readInt(); > > and this would fail: > > number = in.readInt(); > text = in.readUTF(); > > ? > > 2013/2/27 Sandy Ryza : > > Hi Paul, > > > > To do this, you need to make your Dog class implement Hadoop's Writable > > interface, so that it can be serialized to and deserialized from bytes. > > > http://hadoop.apache.org/docs/r1.1.1/api/org/apache/hadoop/io/Writable.html > > > > The methods you implement would look something like this: > > > > public void write(DataOutput out) { > > out.writeDouble(weight); > > out.writeUTF(name); > > out.writeLong(date.toTimeInMillis()); > > } > > > > public void readFields(DataInput in) { > > weight = in.readDouble(); > > name = in.readUTF(); > > date = new Date(in.readLong()); > > } > > > > hope that helps, > > Sandy > > > > On Wed, Feb 27, 2013 at 10:34 AM, Paul van Hoven > > wrote: > >> > >> The output value in the map function is in most examples for hadoop > >> something like this: > >> > >> public static class Map extends Mapper >> outputValue> > >> > >> Normally outputValue is something like Text or IntWriteable. > >> > >> I got a custom class with its own properties like > >> > >> public class Dog { > >> string name; > >> Date birthday; > >> double weight; > >> } > >> > >> Now how would I accomplish the following map function: > >> > >> public static class Map extends Mapper >> Dog> > >> > >> ? > > > > > --e89a8f647ac3fdd7bf04d6ba151a Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable That's right, the date needs to be written and read in the same order.<= br>
On Wed, Feb 27, 2013 at 11:04 AM, Paul va= n Hoven <paul.van.hoven@googlemail.com> wrote:
Great! Thank you.

I guess the order for writing and reading the data this way is
important. I mean, for

out.writeUTF("blabla")
out.writeInt(12)

the following would be correct

text =3D in.readUTF();
number =3D in.readInt();

and this would fail:

number =3D in.readInt();
text =3D in.readUTF();

?

2013/2/27 Sandy Ryza <sandy.r= yza@cloudera.com>:
> Hi Paul,
>
> To do this, you need to make your Dog class implement Hadoop's Wri= table
> interface, so that it can be serialized to and deserialized from bytes= .
> http://hadoop.apache.org/docs/r1.1.1/ap= i/org/apache/hadoop/io/Writable.html
>
> The methods you implement would look something like this:
>
> public void write(DataOutput out) {
> =A0 out.writeDouble(weight);
> =A0 out.writeUTF(name);
> =A0 out.writeLong(date.toTimeInMillis());
> }
>
> public void readFields(DataInput in) {
> =A0 weight =3D in.readDouble();
> =A0 name =3D in.readUTF();
> =A0 date =3D new Date(in.readLong());
> }
>
> hope that helps,
> Sandy
>
> On Wed, Feb 27, 2013 at 10:34 AM, Paul van Hoven
> <paul.van.hoven@go= oglemail.com> wrote:
>>
>> The output value in the map function is in most examples for hadoo= p
>> something like this:
>>
>> public static class Map extends Mapper<LongWritable, Text, outp= utKey,
>> outputValue>
>>
>> Normally outputValue is something like Text or IntWriteable.
>>
>> I got a custom class with its own properties like
>>
>> public class Dog {
>> =A0 =A0string name;
>> =A0 =A0Date birthday;
>> =A0 =A0double weight;
>> }
>>
>> Now how would I accomplish the following map function:
>>
>> public static class Map extends Mapper<LongWritable, Text, IntW= ritable,
>> Dog>
>>
>> ?
>
>

--e89a8f647ac3fdd7bf04d6ba151a--