hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jackob Carlsson <jackob.carls...@gmail.com>
Subject Re: Combiner function
Date Mon, 02 Aug 2010 21:27:52 GMT
Thanks Edward.

> Is there a way we use it for several mappers as well?

> No. That is the exact opposite goal of the combiner. It runs locally.

OK, lets say a stupid scenario, when for instance one mapper is late to
produce the results and it cause a waiting for a reducer task. Then, how to
optimize this case?

> >it may or may not run on a particular map attempt
> It only runs when certain thresholds in the framework are reached.
> http://philippeadjiman.com/blog/2010/01/14/hadoop-tutorial-series-issue-4-to-use-or-not-to-use-a-combiner/

What are these thresholds that may or may not run on a particular map

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message