hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bejoy Ks <bejoy.had...@gmail.com>
Subject Re: Map Reduce slot
Date Thu, 01 Nov 2012 11:43:21 GMT
Hi Udippan

The number of map/reduce tasks slots on your cluster is the sum of all the
slots from your TaskTracker nodes. Based on your node's resource
availability you can even configure that on a per node basis. The slots are
defined on a node level using the following properties
mapred.tasktracker.map.tasks.maximum and
mapred.tasktracker.reduce.tasks.maximum .

As Kapil mentioned the total number of slots across the cluster can be
obtained from JT web UI.

The other 3 properties can be defined on a job level. However in production
clustes the jvm size is marked final to prevent abuses that may lead to
OOMs.

The jvm size of task jvms are defined by 'mapred.child.java.opts' which
defaults to 200 Mb and the jvm reuse is defined by
'mapred.job.reuse.jvm.num.tasks' which defaults to 1 task per jvm.

HTH.

Regards
Bejoy KS

Mime
View raw message