hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "FAQ" by Arun C Murthy
Date Wed, 19 Mar 2008 00:25:16 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The following page has been changed by Arun C Murthy:
http://wiki.apache.org/hadoop/FAQ

The comment on the change is:
Fixed FAQ# 13

------------------------------------------------------------------------------
  [[Anchor(13)]]
  '''13. [#13 MR. I see a maximum of 2 maps/reduces spawned concurrently on each TaskTracker,
how do I increase that?]'''
  
- Use the configuration knob: [http://hadoop.apache.org/core/docs/current/hadoop-default.html#mapred.tasktracker.tasks.maximum
mapred.tasktracker.tasks.maximum] to control the number of maps/reduces spawned simultaneously
on a !TaskTracker. By default, it is set to ''2'', hence one sees a maximum of 2 maps and
2 reduces at a given instance on a !TaskTracker.
+ Use the configuration knob: [http://hadoop.apache.org/core/docs/current/hadoop-default.html#mapred.tasktracker.map.tasks.maximum
mapred.tasktracker.map.tasks.maximum] and [http://hadoop.apache.org/core/docs/current/hadoop-default.html#mapred.tasktracker.reduce.tasks.maximum
mapred.tasktracker.reduce.tasks.maximum] to control the number of maps/reduces spawned simultaneously
on a !TaskTracker. By default, it is set to ''2'', hence one sees a maximum of 2 maps and
2 reduces at a given instance on a !TaskTracker.
  
+ You can set those on a per-tasktracker basis to accurately reflect your hardware (i.e. set
those to higher nos. on a beefier tasktracker etc.).
- Caveats:
-    * ''mapred.tasktracker.tasks.maximum'' that is a cluster-wide limit i.e. controlled at
the !JobTracker end. [http://issues.apache.org/jira/browse/HADOOP-1245 HADOOP-1245] should
fix that.
-    * ''mapred.tasktracker.tasks.maximum'' controls the number of maps '''and''' number of
reduces (independently). [http://issues.apache.org/jira/browse/HADOOP-1274 HADOOP-1274] should
fix that.
- 
  
  [[BR]]
  [[Anchor(14)]]

Mime
View raw message