hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amr Shahin <amrnab...@gmail.com>
Subject Re: Combiner Execution
Date Tue, 22 Oct 2013 15:27:25 GMT
What sort of combining are you trying to achieve?
Hadoop combining means that hadoop will collect the output from all the
maps and guarantee that all the outputs that have the same key will be sent
to the same reducer (you can find more details in "hadoop the definitive
guide chapters 2 and 6).



On Tue, Oct 22, 2013 at 5:50 PM, Mike Schliep <schliep@cs.umn.edu> wrote:

> For a class project my group and I are looking to experiment with
> combining the output from Mappers on the same node or in the same rack. We
> found the idea at http://wiki.apache.org/hadoop/**HadoopResearchProjects<http://wiki.apache.org/hadoop/HadoopResearchProjects>
> .
>
> According to http://developer.yahoo.com/**hadoop/tutorial/module4.html<http://developer.yahoo.com/hadoop/tutorial/module4.html>the
output is already combined over all Mappers in a node. But we can not
> find how this is happening. Can someone point us to where this combiner is
> executed?
>
> Thanks,
> Mike Schliep
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message