hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lin Yang <lin.yang.ja...@gmail.com>
Subject Should I set mapred.tasktracker.map.tasks.maximum = number of processor or number of cores?
Date Wed, 20 Nov 2013 12:42:19 GMT
Hi, all,

I'm running hadoop on a cluster consisting of 2 data nodes, each of which
has 24 CPUs (intel Xeon X5670@2.93G) and each CPU has 6 cores. So, totally
144 cores on a single node.

In this case, what value should I set for these parameters?

   - mapred.tasktracker.map.tasks.maximum
   - mapred.map.tasks
   - mapred.tasktracker.reduce.tasks.maximum
   - mapred.reduce.tasks

Actually, I've searched the answer on Internet, but I've been confused
since some articles said these should be related to #processors and the
others said it should be related to #cores.

Could anyone give me a confirmed formulation to calculate these parameters?


Lin Yang

View raw message