hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael <michael.micel...@gmail.com>
Subject Re: Threads per mapreduce job
Date Sat, 27 Dec 2008 22:23:58 GMT
Thanks Sagar,
However, when I add this to my hadoop-site.xml it doesn't listen:
<property>
  <name>mapred.map.multithreadedrunner.threads</name>
  <value>1</value>
</property>
I added it to both servers and here is the output of a test mapreduce run:
08/12/27 16:09:05 INFO mapred.JobClient:   Job Counters
08/12/27 16:09:05 INFO mapred.JobClient:     Launched reduce tasks=1
08/12/27 16:09:05 INFO mapred.JobClient:     Rack-local map tasks=16
08/12/27 16:09:05 INFO mapred.JobClient:     Launched map tasks=16

Thanks,
Michael

Sagar Naik wrote:
> mapred.map.multithreadedrunner.threads
> is the property u r looking for
> 
> 
> Michael wrote:
>> Hi everyone:
>> How do I control the number of threads per mapreduce job.  I am using
>> bin/hadoop jar wordcount to run jobs and even though I have found these
>> settings in hadoop-default.xml and changed the values to 1:
>> <name>mapred.tasktracker.map.tasks.maximum</name>
>> <name>mapred.tasktracker.reduce.tasks.maximum</name>
>>
>> The output of the job seems to indicate otherwise.
>> 08/12/26 18:21:12 INFO mapred.JobClient:   Job Counters
>> 08/12/26 18:21:12 INFO mapred.JobClient:     Launched reduce tasks=1
>> 08/12/26 18:21:12 INFO mapred.JobClient:     Rack-local map tasks=12
>> 08/12/26 18:21:12 INFO mapred.JobClient:     Launched map tasks=17
>> 08/12/26 18:21:12 INFO mapred.JobClient:     Data-local map tasks=4
>>
>> I have 2 servers running the mapreduce process and the datanode process.
>> Thanks,
>> Michael
>>
>>   
> 

Mime
View raw message