hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Schlosser" <swschlos...@gmail.com>
Subject Re: Using my own classes as keys and values
Date Thu, 11 Oct 2007 01:28:49 GMT
Ah - this was the problem!  Now that I have the constructor, I am able
to serialize either way - using Java's serialization or my own.  For
this app, I'm happy either way, but I'll think about sticking with my
own serialization in the future.

Thanks for the help!

-steve

On 10/10/07, Christopher Douglas <chrisdo@yahoo-inc.com> wrote:
> It looks like you're missing a *public*, default constructor; the
> framework can't create an instance to call readFields() on.
>
> That said: you might want to reconsider using Java serialization within
> Writables. I know it seems restrictive, but you're opening yourself up
> to a rancid cornucopia of ClassLoader, static, etc. issues. -C
>
> > -----Original Message-----
> > From: Steve Schlosser [mailto:swschlosser@gmail.com]
> > Sent: Wednesday, October 10, 2007 7:57 AM
> > To: hadoop-user@lucene.apache.org
> > Subject: Using my own classes as keys and values
> >
> > Hello all
> >
> > Is there a best practice for using my own classes as keys and values?
> >
> > My first attempt at doing this was successful - I built a
> > BigIntegerWritable class using IntWritable as a template.  It was easy
> > because BigInteger has methods converting to and from byte arrays,
> > which I could then write into the DataOutput or read from the
> > DataInput.
> >
> > It seems like I should be able to use object serialization to write
> > to/read from the DataOutput/Input objects and make my own classes
> > implement the Writable interface.  It seems like I should be able to
> > do something like this:
> >
> > import java.io.*;
> >
> > import org.apache.hadoop.io.*;
> >
> > public class Sample implements Writable {
> >
> >     Address address;
> >     SampleValue value;         // sampled value at this point
> >
> >     public Sample(Address a, SampleValue v) {
> >         address = a;
> >         value = v;
> >     }
> >
> >     public SampleValue getValue()  { return value;}
> >     public Address getAddress() { return address; }
> >
> >     public String toString () {
> >         return (address.toString() + " " + value.toString());
> >     }
> >
> > [...]
> >
> >     public void readFields(DataInput in) throws IOException {
> >         ObjectInputStream oin = new
> > ObjectInputStream((DataInputBuffer)in);
> >
> >         try {
> >             address = (Address)oin.readObject();
> >             value = (SampleValue)oin.readObject();
> >         } catch (ClassNotFoundException e) {
> >             throw new IOException(e.toString());
> >         }
> >
> >     }
> >
> >     public void write(DataOutput out) throws IOException {
> >         ObjectOutputStream oout = new
> > ObjectOutputStream((DataOutputBuffer)out);
> >
> >         oout.writeObject(address);
> >         oout.writeObject(value);
> >     }
> > }
> >
> > This code compiles, but throws exceptions at runtime, complaining that
> > WritableComparator can not access a member of class Sample with
> > modifiers "".  Can someone tell me what this exception is talking
> > about?
> >
> > Do I need to implement a WritableComparator for each class that I want
> > to implement Writable?
> >
> > Thanks again for the help.
> >
> > -steve
>

Mime
View raw message