hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Shook <ash...@clearedgeit.com>
Subject FairScheduler Local Task Restriction
Date Thu, 22 Sep 2011 19:41:20 GMT
Hello All,

I have recently switched my small Hadoop dev cluster (v0.20.1) to use the FairScheduler. 
I have a max of 128 map tasks available and recently noticed that my jobs seem to use a maximum
of 16 at any given time (the job I am looking at in particular runs for about 15 minutes)
- they are also all data local map tasks.  I searched around a bit and discovered the mapred.fairscheduler.locality.delay
may be to blame.  I set it to 0 in mapred-site.xml, copied the file around to my nodes and
tried running another job.  It still has 16 tasks.

Does it require a cluster restart?  Is it something totally different?  Should I not set this
value to zero?


-- Adam

View raw message