hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oren Bumgarner <oren...@gmail.com>
Subject Exception during reduce phase when running jobs remotely
Date Fri, 26 Apr 2013 18:50:30 GMT
I have a small hadoop cluster running 1.0.4 and I'm trying to have it setup
so that I can run jobs remotely from a computer on the same network that is
not a part of the cluster. I've got a main java class that
implements org.apache.hadoop.util.Tool and I'm able to run this job from
the NameNode using ToolRunner.run(), setting up the JobConf, and submitting
with JobClient.submitJob().

When I try to run the same class remotely from any machine that is not the
NameNode the job is submitted and it appears that the Map tasks
successfully complete, but I get the following exception for all of the
reduce tasks:

org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
output/map_0.out in any of the configured local directories
	at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
	at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
	at org.apache.hadoop.mapred.MapOutputFile.getInputFile(MapOutputFile.java:161)
	at org.apache.hadoop.mapred.ReduceTask.getMapFiles(ReduceTask.java:220)
	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:398)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
	at org.apache.hadoop.mapred.Child.main(Child.java:249)


I'm not sure how to interpret that error message. I think I'm missing
some config files that are not present on the remote machine but I
can't pin down exactly what I need. Does anyone have any guidance on
what the above error means or how to submit jobs remotely?

Thanks,

--Oren

Mime
View raw message