hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srinath Perera <srin...@wso2.com>
Subject Hadoop fail to find working directory when running Debug Scripts in 1.0.0
Date Sat, 26 May 2012 02:12:36 GMT
Hi All,

I am using Hadoop 1.0.0.

I am trying to write a sample that shows how debug scripts works. To do
that, I have written a map reduce job that always fail (by explicitly
throwing an exception) see FaultyWordCount.java. I have setup the debug
scripts as per http://developer.yahoo.com/hadoop/tutorial/module5.htmlWordcountWithDebugScript.java

When I run the job, I get the following error when Hadoop try to run the
debug script.

-------------------- DEBUG OUT---------------------
bash: /Users/srinath/playground/hadoop-book/hadoop-1.0.0/fail-script: No
such file or directory
bash: line 0: exec:
/Users/srinath/playground/hadoop-book/hadoop-1.0.0/fail-script: cannot
execute: No such file or directory

I looked at further, and error happens due to following error (found from
task tracker logs)

2012-05-26 05:59:57,110 WARN org.apache.hadoop.mapred.TaskTracker: Working
Directory of the task attempt_201205260558_0001_m_000000_0 doesnt exist.
Caught exception org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find
taskTracker/srinath/jobcache/job_201205260558_0001/attempt_201205260558_0001_m_000000_0/work
in any of the configured local directories
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
at
org.apache.hadoop.mapred.TaskTracker$TaskInProgress.taskFinished(TaskTracker.java:2871)
at
org.apache.hadoop.mapred.TaskTracker$TaskInProgress.reportTaskFinished(TaskTracker.java:2772)
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:285)
This happens in line 2871 of org.apache.hadoop.mapred.TaskTracker, where
LocalDirAllocator.getLocalPathToRead( ..) throws an error. As I understood
this happens due to work directory is missing from
taskTracker/srinath/jobcache/job_201205260558_0001/attempt_201205260558_0001_m_000000_0/.
Rest of the path exists.

When is work directory created? is it always created?

Any thoughts on what I am doing wrong?

Thanks
--Srinath


-- 
============================
Srinath Perera, Ph.D.
  Senior Software Architect, WSO2 Inc.
  Visiting Faculty, University of Moratuwa
  Member, Apache Software Foundation
  Research Scientist, Lanka Software Foundation
  Blog: http://srinathsview.blogspot.com/
  Photos: http://www.flickr.com/photos/hemapani/
 Phone: 0772360902

Mime
View raw message