hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bejoy Ks <bejoy.had...@gmail.com>
Subject Re: What determines the map task / reduce task capacity? average task per node?
Date Tue, 03 Apr 2012 08:28:12 GMT
Hi Xin
      Yes, the number of worker nodes do count on the map and reduce
capacity of the cluster. The map and reduce task capacity/slots is
dependen't on each node and of course the requirements of your applications
that use the cluster. Based on the available memory, number of cores etc
you need to configure the slots so that there won't be any resource crunch
while running your tasks. You can set the slots on each node it the
corresponding mapred-site.xml using.

<name>mapred.tasktracker.map.tasks.maximum</name>
<value>12</value>
</property>
<property>
<name>mapred.tasktracker.reducer.tasks.maximum</name>
<value>4</value>
</property>

Regards
Bejoy KS

On Tue, Apr 3, 2012 at 1:45 PM, Fang Xin <nusfangxin@gmail.com> wrote:

> Hi all,
>
> of course it's sensible that number of nodes in the cluster will
> influence map / reduce task capacity, but what determines average task
> per node?
> Can the number be manually set? any hardware constraint on setting the
> number?
>
> Thank you!
> Xin
>

Mime
View raw message