hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rakesh kothari <rkothari_...@hotmail.com>
Subject RE: mapred.local.dir cleanup
Date Thu, 20 Jan 2011 22:59:07 GMT

Any ideas on how "attempt*" directories getting created directly under "mapred.local.dir"
? Pointers to parts of the source code would help too.


From: rkothari_iit@hotmail.com
To: mapreduce-user@hadoop.apache.org
Subject: mapred.local.dir cleanup
Date: Tue, 18 Jan 2011 17:20:04 -0800


I am seeing lots of leftover directories going back as far as 12 days in the task trackers
"mapred.local.dir". These directories are for "M/R task attempts".

How are these directories end up in "mapred.local.dir" as  from my understanding these directories
should be in "mapred.local.dir/taskTracker/jobcache/job-Id/" and should be cleaned up once
the job finishes (or after some interval) ? How can I enable automatic cleanup of these directories

A big chunk of these leftover directories were created the same day/time when I bounced my
hadoop cluster. 

Any pointers is highly appreciated.

View raw message