hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jackob Carlsson <jackob.carls...@gmail.com>
Subject Re: Combiner function
Date Mon, 02 Aug 2010 20:28:59 GMT
Thanks Nick, but "in-memory" means a combiner can only be used over a single
mapper?right?! Is there a way we use it for several mappers as well? Also
what do you mean by "it may or may not run on a particular map attempt"?

Br,
Jackob

On Mon, Aug 2, 2010 at 5:43 PM, Nick Jones <nick.jones@amd.com> wrote:

> Hi Jackob,
> A combiner acts a lot like a reduce step but it's executed on the mapper
> with in-memory data.  I've seen a reduction in job execution time by adding
> one.  The one caveat to keep in mind is that it may or may not run on a
> particular map attempt.
>
> Nick
>
>
>
> On 8/2/2010 10:39 AM, Jackob Carlsson wrote:
>
>> Hi everyone,
>> Could anyone please help me to understand the function of combiner?
>>
>> Thanks in advance
>> Jackob
>>
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message