hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Devaraj K <devara...@huawei.com>
Subject RE: Relation between Mapper and Combiner
Date Fri, 01 Jul 2011 11:47:25 GMT
Hi Lucian,

 

            For every map task, combiner will be executed multiple times
before writing the map output. Combine step is not a separate task and it is
part of map task execution. Reducer will copy the output of the map task
which is reduced by the combiner.

 

>For example:
>If I have 2 map tasks ran on the same machine, will I have 1 combine task
on that machine to combine the maps outputs, or 2 combine tasks?

 

In this case, combiner will be executed for each map task independent to
each other. This combiner step will execute multiple times till it gets same
output for 1 or more runs of combiner.



 

You can go through Combiner section here for more info :
http://wiki.apache.org/hadoop/HadoopMapReduce

 

Devaraj K 

----------------------------------------------------------------------------
---------------------------------------------------------
This e-mail and its attachments contain confidential information from
HUAWEI, which 
is intended only for the person or entity whose address is listed above. Any
use of the 
information contained herein in any way (including, but not limited to,
total or partial 
disclosure, reproduction, or dissemination) by persons other than the
intended 
recipient(s) is prohibited. If you receive this e-mail in error, please
notify the sender by 
phone or email immediately and delete it!ss

 

  _____  

From: Lucian Iordache [mailto:lucian.george.iordache@gmail.com] 
Sent: Friday, July 01, 2011 2:25 PM
To: mapreduce-user@hadoop.apache.org
Subject: Relation between Mapper and Combiner

 

Hello guys,

Can anybody tell me which is the relation between map task and combine
tasks?
I would like to know if there is a 1:1 relation between them, or is a *:1
(many to one).

For example:
If I have 2 map tasks ran on the same machine, will I have 1 combine task on
that machine to combine the maps outputs, or 2 combine tasks?

Best Regards,
-- 
Lucian


Mime
View raw message