hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From William Slacum <wsla...@gmail.com>
Subject Re: Custom comparator when using Kryo serializer for MapReduce serialization
Date Wed, 19 Aug 2015 17:06:41 GMT
In a general sense, if you can guarantee that your objects serialize in
lexicographical order, then you should be able to do a comparator on the
raw bytes themselves without any interpretation.

On Wed, Aug 19, 2015 at 5:21 AM, Yaron Gonen <yaron.gonen@gmail.com> wrote:

> Hi all,
> (I'm using Hadoop 1.2.1)
> I'm using Kryo <https://github.com/EsotericSoftware/kryo> (with chill
> <https://github.com/twitter/chill>) as my serializer (instead of the
> Writable interface).
> However, I'm having trouble with the comparator: on one hand, since none
> of my objects are Writable, I cannot use WritableComparator. On the other
> hand, I can work with the RawComparator, but it means to deserialize the
> bytes array each time - seems not very efficient...
> Is there a way to give just an implementation of Java's Comparator? or to
> make the serialized object Comparable?
>
> Regards,
> Yaron
>

Mime
View raw message