hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yang Li <liy...@cn.ibm.com>
Subject Re: java.io.FileNotFoundException
Date Mon, 10 May 2010 00:17:52 GMT
Yes, I'm using 0.20.2.  I run it in pseudo mode on windows xp + cygwin as 
a private playground.  All default settings with following change:

core-site.xml:
-----------------------------
<configuration>

  <property>
    <name>fs.default.name</name>
    <value>hdfs://localhost:9000</value>
  </property>
  <property>
    <name>hadoop.tmp.dir</name>
    <value>/tmp/hadoop</value>
  </property>

</configuration>


hdfs-site.xml
-----------------------------
<configuration>

  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>

</configuration>


mapred-site.xml
----------------------------
<configuration>

  <property>
    <name>mapred.job.tracker</name>
    <value>localhost:9002</value>
  </property>
  <property>
    <name>mapred.tasktracker.map.tasks.maximum</name>
    <value>1</value>
  </property>
  <property>
    <name>mapred.child.tmp</name>
     <value>/tmp/hadoop/mapred/mapred.child.tmp</value>
  </property>

</configuration>


These can run wordcount example at least, like `hadoop/bin/hadoop jar 
hadoop/hadoop-0.20.2-examples.jar wordcount input output`.  I never tried 
a distributed cluster on windows, you'll definitely go linux if decided to 
do something more serious.


Best regards,
-----------------------
Li Yang, ext. 22056, User Technologies Development, Shanghai, China




Carlos Eduardo Moreira dos Santos <cems@cemshost.com.br> 
Sent by: cemsbr@gmail.com
2010-05-10 01:13
Please respond to
common-user@hadoop.apache.org


To
common-user@hadoop.apache.org
cc

Subject
Re: java.io.FileNotFoundException






I've already tried that. Are you using 0.20.2?

Thank you,
Carlos Eduardo

On Sun, May 9, 2010 at 8:34 AM, Yang Li <liyang@cn.ibm.com> wrote:

> I met the same problem on window+cygwin, don't know the root cause, but 
it
> can be worked around by a explicit "mapred.child.tmp" property.  Try add
> these lines to your <conf>/mapred-site.xml:
>
>  <property>
>    <name>mapred.child.tmp</name>
>     <value>/tmp/hadoop/mapred/mapred.child.tmp</value>
>  </property>
>
> The examples then worked on my laptop.
>
> Best regards,
> -----------------------
> Li Yang, ext. 22056, User Technologies Development, Shanghai, China
>
>
>
>
> Carlos Eduardo Moreira dos Santos <cems@cemshost.com.br>
> Sent by: cemsbr@gmail.com
> 2010-05-02 12:42
> Please respond to
> common-user@hadoop.apache.org
>
>
> To
> common-user <common-user@hadoop.apache.org>
> cc
>
> Subject
> java.io.FileNotFoundException
>
>
>
>
>
>
> Hadoop is working fine in Linux. In Windows (using cygwin) I can't get
> mapred to work, though hdfs is ok. This is the stacktrace:
>
> java.io.FileNotFoundException: File
>
> 
E:/tmp/hadoop-SYSTEM/mapred/local/taskTracker/jobcache/job_201005020105_0001/attempt_201005020105_0001_m_000002_0/work/tmp
> does not exist.
>                 at
>
> 
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361)
>                 at
>
> 
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
>                 at
> org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519)
>                 at org.apache.hadoop.mapred.Child.main(Child.java:155)
>
> The beginning of the path (E:/) seems strange (I was hoping for
> something like /cygdrive/e or just /tmp). I read TaskRunner.java:519
> and I tried to use an absolute path in mapred.child.tmp in
> conf/mapred-site.xml, but it keeps looking for the same path even if I
> restart mapred. The path exists until jobcache
>
> The tasktracker log shows:
>
> 2010-05-02 01:06:23,765 INFO org.apache.hadoop.mapred.TaskTracker:
> LaunchTaskAction (registerTask): attempt_201005020105_0001_m_000002_0
> task's state:UNASSIGNED^M
> 2010-05-02 01:06:23,765 INFO org.apache.hadoop.mapred.TaskTracker:
> LaunchTaskAction (registerTask): attempt_201005020105_0001_m_000003_0
> task's state:UNASSIGNED^M
> 2010-05-02 01:06:23,765 INFO org.apache.hadoop.mapred.TaskTracker:
> Trying to launch : attempt_201005020105_0001_m_000002_0^M
> 2010-05-02 01:06:23,765 INFO org.apache.hadoop.mapred.TaskTracker: In
> TaskLauncher, current free slots : 2 and trying to launch
> attempt_201005020105_0001_m_000002_0^M
> 2010-05-02 01:06:37,562 INFO org.apache.hadoop.mapred.TaskTracker:
> Trying to launch : attempt_201005020105_0001_m_000003_0^M
> 2010-05-02 01:06:37,562 INFO org.apache.hadoop.mapred.TaskTracker: In
> TaskLauncher, current free slots : 1 and trying to launch
> attempt_201005020105_0001_m_000003_0^M
> 2010-05-02 01:06:37,625 INFO org.apache.hadoop.mapred.JvmManager: In
> JvmRunner constructed JVM ID: jvm_201005020105_0001_m_518928642^M
> 2010-05-02 01:06:37,625 INFO org.apache.hadoop.mapred.JvmManager: JVM
> Runner jvm_201005020105_0001_m_518928642 spawned.^M
> 2010-05-02 01:06:37,921 INFO org.apache.hadoop.mapred.JvmManager: In
> JvmRunner constructed JVM ID: jvm_201005020105_0001_m_1918177803^M
> 2010-05-02 01:06:37,921 INFO org.apache.hadoop.mapred.JvmManager: JVM
> Runner jvm_201005020105_0001_m_1918177803 spawned.^M
> 2010-05-02 01:06:40,312 INFO org.apache.hadoop.mapred.TaskTracker: JVM
> with ID: jvm_201005020105_0001_m_518928642 given task:
> attempt_201005020105_0001_m_000002_0^M
> 2010-05-02 01:06:40,578 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201005020105_0001_m_000002_0 0.0% ^M
> 2010-05-02 01:06:40,687 INFO org.apache.hadoop.mapred.JvmManager: JVM
> : jvm_201005020105_0001_m_518928642 exited. Number of tasks it ran:
> 0^M
> 2010-05-02 01:06:41,046 INFO org.apache.hadoop.mapred.TaskTracker: JVM
> with ID: jvm_201005020105_0001_m_1918177803 given task:
> attempt_201005020105_0001_m_000003_0^M
> 2010-05-02 01:06:41,265 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201005020105_0001_m_000003_0 0.0% ^M
> 2010-05-02 01:06:41,421 INFO org.apache.hadoop.mapred.JvmManager: JVM
> : jvm_201005020105_0001_m_1918177803 exited. Number of tasks it ran:
> 0^M
> 2010-05-02 01:06:43,687 INFO org.apache.hadoop.mapred.TaskRunner:
> attempt_201005020105_0001_m_000002_0 done; removing files.^M
> 2010-05-02 01:06:43,687 INFO org.apache.hadoop.mapred.TaskTracker:
> addFreeSlot : current free slots : 1^M
> 2010-05-02 01:06:44,421 INFO org.apache.hadoop.mapred.TaskRunner:
> attempt_201005020105_0001_m_000003_0 done; removing files.^M
> 2010-05-02 01:06:44,421 INFO org.apache.hadoop.mapred.TaskTracker:
> addFreeSlot : current free slots : 2^M
> 2010-05-02 01:07:20,953 INFO org.apache.hadoop.mapred.TaskTracker:
> Received 'KillJobAction' for job: job_201005020105_0001
>
> Any idea?
>
> Thank you,
> Carlos Eduardo
>
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message