hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley" <owen.omal...@gmail.com>
Subject Re: Values returned by Map to Reducer
Date Tue, 17 Nov 2009 03:47:43 GMT
On Mon, Nov 16, 2009 at 11:43 AM, Something Something <
mailinglists19@gmail.com> wrote:

> Does Hadoop Mapreduce guarantee that the *values* returned by Mapper to the
> Reducer are sorted?  Can I safely assume that?  Would it always be true -
> at
> least for 'Text' type?
>
> public void reduce(Text key, *Iterable<Text> values*, Context context
>

No, the values will *not* be sorted. In fact, it will be non-deterministic
between multiple runs of the job with the same input. The keys will always
be sorted. If you want the values to be sorted, you need to take additional
steps. Please look at the SecondarySort example. It shows exactly how to get
the values sorted in the order you desire.

-- Owen

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message