hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 심탁길 <1004...@nhncorp.com>
Subject Initial number of Maps on the machine
Date Mon, 19 Mar 2007 02:13:27 GMT
When I run Simple MR Job such as grep (about 200Maps & 4 Reduces) with 20 Opeteron Servers
( 2Way Dual-Core, 4GB RAM)

Only 2 maps are instatantiated on one machine and each map task takes 5~6 seconds to be done.
 

As a result, about 50% CPU is unsed during the MR Job and the overall performance is not that
good as I expected 

Configuration "mapred.tasktracker.tasks.maximum ==> 10", It only works when each map tasks
last more than 10 seconds

It seemds that Hadoop framework starts MR job with the limit of 2 maps on one machine.  

When running two similar MR Jobs concurrently, the number of maps on one machine is still
2 and CPU usage is about 50% and each MR Jobs takes almost 2times longer to be done.

Then, How can I change the initial limit of map's count on the machine ?

 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message