hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Is the input values for reduce method sorted in any order?
Date Tue, 26 Mar 2013 06:28:30 GMT
MR will partition and sort inputs by keys by the key comparator, and
then group them together when reading back via a grouping comparator
(which is usually the same as the key comparator). It will not re-sort
the values nor look at any of the value's fields during this process.

If you want your values to be sorted, implementing secondary sort
would need to happen wherein you extract the value-sort attributes
onto your keys and have them grouped differently.

On Tue, Mar 26, 2013 at 10:55 AM, jingguo yao <yaojingguo@gmail.com> wrote:
> Is the Iterable values associated with a key sorted in any order?  Are
> there are any configuration options controlling how the input values are
> sorted?
> I know that the secondary sort way can be used to achieve the same
> effect. I am not asking for a workaround.
> --
> Jingguo

Harsh J

View raw message