hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Kendall <mkend...@justin.tv>
Subject Re: log.tmp does not exist, running hadoop in pseudo distributed
Date Thu, 12 Nov 2009 01:40:32 GMT
My first guess is that your tmp directory isn't set up correctly.  Also, I
don't know about WordCountv2 but the original wordcount needed an output
directory passed along with an input directory (the directory has to not
exist when you start the job).

-mike

On Wed, Nov 11, 2009 at 5:33 PM, Ahmad Ali Iqbal
<ahmad.ali.iqbal@gmail.com>wrote:

> Hi all,
>
> I am a new user of hadoop and trying to run the WordCount v2 example in a
> pseudo distributed operation given at
> http://hadoop.apache.org/common/docs/current/mapred_tutorial.html<
> http://hadoop.apache.org/common/docs/current/mapred_tutorial.html#Example%3A+WordCount+v2.0
> >but
> when I run it I get errors saying log.tmp does not exist. In order to
> setup pseudo distributed mode, I have followed in instructions from
> http://hadoop.apache.org/common/docs/current/quickstart.html and modified
> the configuration files. ssh localhost is working fine (but in order to run
> this program, I tested in both ways; by connecting localhost through ssh
> and
> without connecting it). Please see the output as follows
>
> ahmad@aai:~/hadoop-dev$ ls
> bin  dfs  logs  wcv1  wcv2  WordCountv1.java  WordCountv2.java
> ahmad@aai:~/hadoop-dev$ ls -l wcv2/
> total 24
> drwxr-xr-x 2 ahmad ahmad 4096 2009-11-11 14:47 input
> drwxr-xr-x 2 ahmad ahmad 4096 2009-11-09 16:05 output
> -rw-r--r-- 1 ahmad ahmad   12 2009-11-03 15:08 patterns.txt
> drwxr-xr-x 2 ahmad ahmad 4096 2009-11-03 13:47 wordcount_classes
> -rw-r--r-- 1 ahmad ahmad 5536 2009-11-06 10:37 wordcount.jar
> ahmad@aai:~/hadoop-dev$ bin/hadoop namenode -format
> 09/11/11 14:49:13 INFO namenode.NameNode: STARTUP_MSG:
> /*****************************
> *******************************
> STARTUP_MSG: Starting NameNode
> STARTUP_MSG:   host = aai/127.0.1.1
> STARTUP_MSG:   args = [-format]
> STARTUP_MSG:   version = 0.20.1
> STARTUP_MSG:   build =
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.20.1-rc1 -r
> 810220; compiled by 'oom' on Tue Sep  1 20:55:56 UTC 2009
> ************************************************************/
> 09/11/11 14:49:14 INFO namenode.FSNamesystem:
> fsOwner=ahmad,ahmad,adm,dialout,cdrom,plugdev,lpadmin,admin,sambashare
> 09/11/11 14:49:14 INFO namenode.FSNamesystem: supergroup=supergroup
> 09/11/11 14:49:14 INFO namenode.FSNamesystem: isPermissionEnabled=true
> 09/11/11 14:49:14 INFO common.Storage: Image file of size 95 saved in 0
> seconds.
> 09/11/11 14:49:14 INFO common.Storage: Storage directory
> dfs/hadoop-ahmad/dfs/name has been successfully formatted.
> 09/11/11 14:49:14 INFO namenode.NameNode: SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down NameNode at aai/127.0.1.1
> ************************************************************/
> ahmad@aai:~/hadoop-dev$ bin/start-all.sh
> starting namenode, logging to
> /home/ahmad/hadoop-dev/bin/../logs/hadoop-ahmad-namenode-aai.out
> ahmad@localhost's password:
> localhost: starting datanode, logging to
> /home/ahmad/hadoop-dev/bin/../logs/hadoop-ahmad-datanode-aai.out
> ahmad@localhost's password:
> localhost: starting secondarynamenode, logging to
> /home/ahmad/hadoop-dev/bin/../logs/hadoop-ahmad-secondarynamenode-aai.out
> starting jobtracker, logging to
> /home/ahmad/hadoop-dev/bin/../logs/hadoop-ahmad-jobtracker-aai.out
> ahmad@localhost's password:
> localhost: starting tasktracker, logging to
> /home/ahmad/hadoop-dev/bin/../logs/hadoop-ahmad-tasktracker-aai.out
> ahmad@aai:~/hadoop-dev$ bin/hadoop fs -put wcv2/input input
> ahmad@aai:~/hadoop-dev$ bin/hadoop jar wcv2/wordcount.jar WordCountv2
> input
> output
> 09/11/11 14:52:14 INFO mapred.FileInputFormat: Total input paths to process
> : 2
> 09/11/11 14:52:14 INFO mapred.JobClient: Running job: job_200911111450_0001
> 09/11/11 14:52:15 INFO mapred.JobClient:  map 0% reduce 0%
> 09/11/11 14:52:22 INFO mapred.JobClient: Task Id :
> attempt_200911111450_0001_m_000004_0, Status : FAILED
> java.io.FileNotFoundException: File
>
> /home/ahmad/hadoop-dev/logs/userlogs/attempt_200911111450_0001_m_000004_0/log.tmp
> does not exist.
>    at
>
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361)
>    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:192)
>    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:142)
>    at
> org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:253)
>    at
> org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:406)
>    at org.apache.hadoop.mapred.TaskLog.writeToIndexFile(TaskLog.java:193)
>    at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:230)
>    at org.apache.hadoop.mapred.Child.main(Child.java:138)
>
> 09/11/11 14:52:23 WARN mapred.JobClient: Error reading task
>
> outputhttp://aai:50060/tasklog?plaintext=true&taskid=attempt_200911111450_0001_m_000004_0&filter=stdout
> 09/11/11 14:52:23 WARN mapred.JobClient: Error reading task
>
> outputhttp://aai:50060/tasklog?plaintext=true&taskid=attempt_200911111450_0001_m_000004_0&filter=stderr
> 09/11/11 14:52:29 INFO mapred.JobClient: Task Id :
> attempt_200911111450_0001_m_000004_1, Status : FAILED
> java.io.FileNotFoundException: File
>
> /home/ahmad/hadoop-dev/logs/userlogs/attempt_200911111450_0001_m_000004_1/log.tmp
> does not exist.
>    at
>
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361)
>    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:192)
>    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:142)
>    at
> org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:253)
>    at
> org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:406)
>    at org.apache.hadoop.mapred.TaskLog.writeToIndexFile(TaskLog.java:193)
>    at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:230)
>    at org.apache.hadoop.mapred.Child.main(Child.java:138)
>
> 09/11/11 14:52:29 WARN mapred.JobClient: Error reading task
>
> outputhttp://aai:50060/tasklog?plaintext=true&taskid=attempt_200911111450_0001_m_000004_1&filter=stdout
> 09/11/11 14:52:29 WARN mapred.JobClient: Error reading task
>
> outputhttp://aai:50060/tasklog?plaintext=true&taskid=attempt_200911111450_0001_m_000004_1&filter=stderr
> 09/11/11 14:52:35 INFO mapred.JobClient: Task Id :
> attempt_200911111450_0001_m_000004_2, Status : FAILED
> java.io.FileNotFoundException: File
>
> /home/ahmad/hadoop-dev/logs/userlogs/attempt_200911111450_0001_m_000004_2/log.tmp
> does not exist.
>    at
>
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361)
>    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:192)
>    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:142)
>    at
> org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:253)
>    at
> org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:406)
>    at org.apache.hadoop.mapred.TaskLog.writeToIndexFile(TaskLog.java:193)
>    at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:230)
>    at org.apache.hadoop.mapred.Child.main(Child.java:138)
>
> 09/11/11 14:52:35 WARN mapred.JobClient: Error reading task
>
> outputhttp://aai:50060/tasklog?plaintext=true&taskid=attempt_200911111450_0001_m_000004_2&filter=stdout
> 09/11/11 14:52:35 WARN mapred.JobClient: Error reading task
>
> outputhttp://aai:50060/tasklog?plaintext=true&taskid=attempt_200911111450_0001_m_000004_2&filter=stderr
> 09/11/11 14:52:47 INFO mapred.JobClient: Task Id :
> attempt_200911111450_0001_m_000003_0, Status : FAILED
> java.io.FileNotFoundException: File
>
> /home/ahmad/hadoop-dev/logs/userlogs/attempt_200911111450_0001_m_000003_0/log.tmp
> does not exist.
>    at
>
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361)
>    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:192)
>    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:142)
>    at
> org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:253)
>    at
> org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:406)
>    at org.apache.hadoop.mapred.TaskLog.writeToIndexFile(TaskLog.java:193)
>    at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:230)
>    at org.apache.hadoop.mapred.Child.main(Child.java:138)
>
> 09/11/11 14:52:47 WARN mapred.JobClient: Error reading task
>
> outputhttp://aai:50060/tasklog?plaintext=true&taskid=attempt_200911111450_0001_m_000003_0&filter=stdout
> 09/11/11 14:52:47 WARN mapred.JobClient: Error reading task
>
> outputhttp://aai:50060/tasklog?plaintext=true&taskid=attempt_200911111450_0001_m_000003_0&filter=stderr
> 09/11/11 14:52:53 INFO mapred.JobClient: Task Id :
> attempt_200911111450_0001_m_000003_1, Status : FAILED
> java.io.FileNotFoundException: File
>
> /home/ahmad/hadoop-dev/logs/userlogs/attempt_200911111450_0001_m_000003_1/log.tmp
> does not exist.
>    at
>
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361)
>    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:192)
>    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:142)
>    at
> org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:253)
>    at
> org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:406)
>    at org.apache.hadoop.mapred.TaskLog.writeToIndexFile(TaskLog.java:193)
>    at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:230)
>    at org.apache.hadoop.mapred.Child.main(Child.java:138)
>
> 09/11/11 14:52:53 WARN mapred.JobClient: Error reading task
>
> outputhttp://aai:50060/tasklog?plaintext=true&taskid=attempt_200911111450_0001_m_000003_1&filter=stdout
> 09/11/11 14:52:53 WARN mapred.JobClient: Error reading task
>
> outputhttp://aai:50060/tasklog?plaintext=true&taskid=attempt_200911111450_0001_m_000003_1&filter=stderr
> 09/11/11 14:52:59 INFO mapred.JobClient: Task Id :
> attempt_200911111450_0001_m_000003_2, Status : FAILED
> java.io.FileNotFoundException: File
>
> /home/ahmad/hadoop-dev/logs/userlogs/attempt_200911111450_0001_m_000003_2/log.tmp
> does not exist.
>    at
>
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361)
>    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:192)
>    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:142)
>    at
> org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:253)
>    at
> org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:406)
>    at org.apache.hadoop.mapred.TaskLog.writeToIndexFile(TaskLog.java:193)
>    at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:230)
>    at org.apache.hadoop.mapred.Child.main(Child.java:138)
>
> 09/11/11 14:52:59 WARN mapred.JobClient: Error reading task
>
> outputhttp://aai:50060/tasklog?plaintext=true&taskid=attempt_200911111450_0001_m_000003_2&filter=stdout
> 09/11/11 14:52:59 WARN mapred.JobClient: Error reading task
>
> outputhttp://aai:50060/tasklog?plaintext=true&taskid=attempt_200911111450_0001_m_000003_2&filter=stderr
> 09/11/11 14:53:04 INFO mapred.JobClient: Job complete:
> job_200911111450_0001
> 09/11/11 14:53:04 INFO mapred.JobClient: Counters: 0
> Exception in thread "main" java.io.IOException: Job failed!
>    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
>    at WordCountv2.run(WordCountv2.java:116)
>    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>    at WordCountv2.main(WordCountv2.java:121)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> ahmad@aai:~/hadoop-dev$
>
>
> Would be grateful for your help.
>
>
> Regards,
> Ahmad
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message