hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saptarshi Guha <saptarshi.g...@gmail.com>
Subject Is the sort(in sort and shuffle) always required
Date Sat, 19 Jun 2010 16:16:01 GMT
Hello,
My question: is the sort (in the sort and shuffle) absolutely required?
If I wanted mapreduce to partition (using the map) and then aggregate(using
reduce) without a need for the keys to be sorted
is it possible to turn of the sorting? Or is the fact that keys come to the
reducer in sorted order just a side effect of sorting and that
the sorting is vital for the efficient operation of MapReduce?


Thanks
Saptarshi

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message