hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley" <omal...@apache.org>
Subject Re: Problem with custom WritableComparable
Date Fri, 12 Nov 2010 15:27:34 GMT
On Thu, Nov 11, 2010 at 4:29 PM, Aaron Baff <Aaron.Baff@telescope.tv> wrote:

> I'm having a problem with a custom WritableComparable that I created to use
> as a Key object. I basically have a number of identifier's with a timestamp,
> and I'm wanting to group the Identifier's together in the reducer, and order
> the records by the timestamp (oldest to newest)

The reduce is called for each distinct key. Fortunately, there is an option
to get different grouping going into the reduce called the "grouping"
comparator. Look at the SecondarySort example for how to do it. Also note
that your partitioner needs to make sure that the partition is only picked
based on the primary key. (This can be effected by making the hashcode only
depend on it, if you use the HashPartitioner.

-- Owen

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message