hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "hao.wang" <hao.w...@ipinyou.com>
Subject Re: Re: how to set mapred.tasktracker.map.tasks.maximum and mapred.tasktracker.reduce.tasks.maximum
Date Tue, 10 Jan 2012 07:09:23 GMT
Hi , 

Thanks for your reply!
According to your suggestion, Maybe I can't apply it to our hadoop cluster.
Cus, each server in our hadoop cluster just contains 2 CPUs. 
     So, I think maybe you mean the core #  but not CPU # in each searver? 
I am looking for your reply.

regards!


2012-01-10 



hao.wang 



发件人: Harsh J 
发送时间: 2012-01-10  11:33:38 
收件人: common-user 
抄送: 
主题: Re: how to set mapred.tasktracker.map.tasks.maximum and mapred.tasktracker.reduce.tasks.maximum

 
Hello again,
Try a 4:3 ratio between maps and reduces, against a total # of available CPUs per node (minus
one or two, for DN and HBase if you run those). Then tweak it as you go (more map-only loads
or more map-reduce loads, that depends on your usage, and you can tweak the ratio accordingly
over time -- changing those props do not need JobTracker restarts, just TaskTracker).
On 10-Jan-2012, at 8:17 AM, hao.wang wrote:
> Hi,
>    Thanks for your reply!
>    I had already read the pages before, can you give me sme more specific suggestions
about how to choose the values of  mapred.tasktracker.map.tasks.maximum and mapred.tasktracker.reduce.tasks.maximum
according to our cluster configuration if possible?
> 
> regards!
> 
> 
> 2012-01-10 
> 
> 
> 
> hao.wang 
> 
> 
> 
> 发件人: Harsh J 
> 发送时间: 2012-01-09  23:19:21 
> 收件人: common-user 
> 抄送: 
> 主题: Re: how to set mapred.tasktracker.map.tasks.maximum and mapred.tasktracker.reduce.tasks.maximum

> 
> Hi,
> Please read http://hadoop.apache.org/common/docs/current/single_node_setup.html to learn
how to configure Hadoop using the various *-site.xml configuration files, and then follow
http://hadoop.apache.org/common/docs/current/cluster_setup.html to achieve optimal configs
for your cluster.
> On 09-Jan-2012, at 5:50 PM, hao.wang wrote:
>> Hi ,all
>>   Our hadoop cluster has 22 nodes including one namenode, one jobtracker and 20 datanodes.
>>   Each node has 2 * 12 cores with 32G RAM
>>   Dose anyone tell me how to config following parameters:
>>   mapred.tasktracker.map.tasks.maximum
>>   mapred.tasktracker.reduce.tasks.maximum
>> 
>> regards!
>> 2012-01-09 
>> 
>> 
>> 
>> hao.wang 
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message