hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sudhakara st <sudhakara...@gmail.com>
Subject Re: Exception during reduce phase when running jobs remotely
Date Sat, 27 Apr 2013 19:35:01 GMT
Hello,

Check you hadoop.tmp.dir and mapred.local.dir configuration and permissions.




On Sat, Apr 27, 2013 at 12:20 AM, Oren Bumgarner <orenbum@gmail.com> wrote:

> I have a small hadoop cluster running 1.0.4 and I'm trying to have it
> setup so that I can run jobs remotely from a computer on the same network
> that is not a part of the cluster. I've got a main java class that
> implements org.apache.hadoop.util.Tool and I'm able to run this job from
> the NameNode using ToolRunner.run(), setting up the JobConf, and submitting
> with JobClient.submitJob().
>
> When I try to run the same class remotely from any machine that is not the
> NameNode the job is submitted and it appears that the Map tasks
> successfully complete, but I get the following exception for all of the
> reduce tasks:
>
> org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find output/map_0.out
in any of the configured local directories
> 	at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
> 	at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
> 	at org.apache.hadoop.mapred.MapOutputFile.getInputFile(MapOutputFile.java:161)
> 	at org.apache.hadoop.mapred.ReduceTask.getMapFiles(ReduceTask.java:220)
> 	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:398)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:249)
>
>
> I'm not sure how to interpret that error message. I think I'm missing some config files
that are not present on the remote machine but I can't pin down exactly what I need. Does
anyone have any guidance on what the above error means or how to submit jobs remotely?
>
> Thanks,
>
> --Oren
>
>


-- 

Regards,
.....  Sudhakara.st

Mime
View raw message