hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: multiple threads
Date Wed, 22 Mar 2006 23:13:40 GMT
Andrew McNabb wrote:
> Does Hadoop run multiple threads on a single slave?

Yes, although each task runs in a separate JVM, not a thread.  A slave 
will run up to mapred.tasktracker.tasks.maximum map and/or reduce tasks 
at a time.

> I have mapred.map.tasks set to 7 (this should be enough to see two jobs
> on each client, though I plan on eventually setting it higher).
> mapred.tasktracker.tasks.maximum is 3 (eventually this will be higher,
> too).

The actual number of map tasks is determined by the number of input 
splits.  Perhaps your input data is not big enough to result in more 
than a few input splits?  A SequenceFile-format input cannot be split 
into chunks smaller than 2k bytes.


View raw message