hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Toby DiPasquale" <codeslin...@gmail.com>
Subject Re: Sorting values of a key in reduce phase
Date Wed, 08 Aug 2007 00:04:47 GMT
On 8/7/07, novice user <pallavip.05@gmail.com> wrote:
> Hi,
>    In reduce phase, with outputValueGroupingComparator, we can sort all keys
> and then group values of a particular key together and send it to reduce()
> method. Is there a way to sort values of a particular key efficiently before
> it reaches to reduce method?

I'm not sure if this is what you want, but Google's MapReduce
framework has the concept of an optional second key parameter for
subsorting of records (saw it in some slides by Jeff Dean). Perhaps
you could integrate this into Hadoop as a patch and submit it?

Toby DiPasquale

View raw message