hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: How to find out what file Hadoop is looking for
Date Tue, 03 Apr 2012 16:12:06 GMT
Some of your TaskTrackers' mapred.local.dirs do not have proper r/w
permissions set on them. Make sure they are owned by the user that
runs the TT service and have read/write permission at least for that
user.

On Tue, Apr 3, 2012 at 6:58 PM, Bas Hickendorff
<hickendorffbas@gmail.com> wrote:
> Hello all,
>
> My map-reduce operation on Hadoop (running on Debian) is correctly
> starting and finding the input file. However, just after starting the
> map reduce, Hadoop tells me that it cannot find a file. Unfortunately,
> it does not state what file it cannot find, or where it is looking.
> Does someone now about what file error is? See below for the complete
> error.
>
> Since the java error is in the chmod() function (judging from the
> stack in the output), I assume it is a problem with the rights, but
> how do I know what rights to change if it gives me no path?
>
> Thanks in advance,
>
> Bas
>
>
>
>
> The output of the job:
>
>
> hadoopmachine@debian:~$ ./hadoop-1.0.1/bin/hadoop jar
> hadooptest/main.jar nl.mydomain.hadoop.debian.test.Main
> /user/hadoopmachine/input /user/hadoopmachine/output
> Warning: $HADOOP_HOME is deprecated.
>
> 12/04/03 08:05:08 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the
> same.
> ****hdfs://localhost:9000/user/hadoopmachine/input
> 12/04/03 08:05:08 INFO input.FileInputFormat: Total input paths to process : 1
> 12/04/03 08:05:08 INFO mapred.JobClient: Running job: job_201204030722_0004
> 12/04/03 08:05:09 INFO mapred.JobClient:  map 0% reduce 0%
> 12/04/03 08:05:13 INFO mapred.JobClient: Task Id :
> attempt_201204030722_0004_m_000002_0, Status : FAILED
> Error initializing attempt_201204030722_0004_m_000002_0:
> ENOENT: No such file or directory
>        at org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method)
>        at org.apache.hadoop.fs.FileUtil.execSetPermission(FileUtil.java:692)
>        at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:647)
>        at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
>        at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
>        at org.apache.hadoop.mapred.JobLocalizer.initializeJobLogDir(JobLocalizer.java:239)
>        at org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:196)
>        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1226)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at javax.security.auth.Subject.doAs(Subject.java:416)
>        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
>        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1201)
>        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1116)
>        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2404)
>        at java.lang.Thread.run(Thread.java:636)
>
> 12/04/03 08:05:13 WARN mapred.JobClient: Error reading task
> outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201204030722_0004_m_000002_0&filter=stdout
> 12/04/03 08:05:13 WARN mapred.JobClient: Error reading task
> outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201204030722_0004_m_000002_0&filter=stderr



-- 
Harsh J

Mime
View raw message