hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: Combiner function
Date Mon, 02 Aug 2010 15:53:28 GMT
As others have pointed out, its mostly applied as an optimization
step. In most cases one's 'Mapper' outputs carry at least a small
group of similar keys that go on to the reducer after a copy and a
sort phase. To reduce it locally (in-memory) via a 'Combiner' helps
reduce data in the copy-sort stages until the 'Reducer' operation

Do note that, implementation-wise, a 'combiner' class must always
collect the same key-value pair types as the mapper function.

On Mon, Aug 2, 2010 at 9:09 PM, Jackob Carlsson
<jackob.carlsson@gmail.com> wrote:
> Hi everyone,
> Could anyone please help me to understand the function of combiner?
> Thanks in advance
> Jackob

Harsh J

View raw message