hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adeel Qureshi <adeelmahm...@gmail.com>
Subject Re: custom writablecomparable with complex fields
Date Sun, 01 Sep 2013 15:43:14 GMT
Okay that makes sense .. so the same order I write is how I can read ..
taking it a step further, in the compareto method, how can I use the bytes
provided to do a comparison on let's say on a list object
On Aug 31, 2013 4:52 PM, "Harsh J" <harsh@cloudera.com> wrote:

> The idea behind write(…) and readFields(…) is simply that of ordering.
> You need to write your custom objects (i.e. a representation of them)
> in order, and read them back in the same order.
>
> An example way of serializing a list would be to first serialize the
> length (so you know how many you'll be needed to read back), and then
> serialize each item appropriately, using delimiters, or using
> length-prefixes just like lists.
>
> Mainly, you're required to tackle the serialization/deserialization on
> your own.
>
> This is one of the reasons I highly recommend using a library like
> Apache Avro instead. Its more powerful, faster, and yet simple to use:
> http://avro.apache.org/docs/current/gettingstartedjava.html and
> http://avro.apache.org/docs/current/mr.html. It is also popular and
> carries first-grade support on several other hadoop-ecosystem
> projects, such as Flume and Crunch.
>
> On Sun, Sep 1, 2013 at 1:23 AM, Adeel Qureshi <adeelmahmood@gmail.com>
> wrote:
> > I want to write a custom writablecomparable object with two List objects
> > within it ..
> >
> > public class CompositeKey implements WritableComparable {
> >
> > private List<JsonKey> groupBy;
> > private List<JsonKey> sortBy;
> > ...
> > }
> >
> > what I am not sure about is how to write
> >
> > readFields and write methods for this object. Any help would be
> appreciated.
> >
> > Thanks
> > Adeel
>
>
>
> --
> Harsh J
>

Mime
View raw message