hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul van Hoven <paul.van.ho...@googlemail.com>
Subject Re: Custom output value for map function
Date Wed, 27 Feb 2013 19:04:07 GMT
Great! Thank you.

I guess the order for writing and reading the data this way is
important. I mean, for

out.writeUTF("blabla")
out.writeInt(12)

the following would be correct

text = in.readUTF();
number = in.readInt();

and this would fail:

number = in.readInt();
text = in.readUTF();

?

2013/2/27 Sandy Ryza <sandy.ryza@cloudera.com>:
> Hi Paul,
>
> To do this, you need to make your Dog class implement Hadoop's Writable
> interface, so that it can be serialized to and deserialized from bytes.
> http://hadoop.apache.org/docs/r1.1.1/api/org/apache/hadoop/io/Writable.html
>
> The methods you implement would look something like this:
>
> public void write(DataOutput out) {
>   out.writeDouble(weight);
>   out.writeUTF(name);
>   out.writeLong(date.toTimeInMillis());
> }
>
> public void readFields(DataInput in) {
>   weight = in.readDouble();
>   name = in.readUTF();
>   date = new Date(in.readLong());
> }
>
> hope that helps,
> Sandy
>
> On Wed, Feb 27, 2013 at 10:34 AM, Paul van Hoven
> <paul.van.hoven@googlemail.com> wrote:
>>
>> The output value in the map function is in most examples for hadoop
>> something like this:
>>
>> public static class Map extends Mapper<LongWritable, Text, outputKey,
>> outputValue>
>>
>> Normally outputValue is something like Text or IntWriteable.
>>
>> I got a custom class with its own properties like
>>
>> public class Dog {
>>    string name;
>>    Date birthday;
>>    double weight;
>> }
>>
>> Now how would I accomplish the following map function:
>>
>> public static class Map extends Mapper<LongWritable, Text, IntWritable,
>> Dog>
>>
>> ?
>
>

Mime
View raw message