hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "#YONG YONG CHENG#" <aarnc...@pmail.ntu.edu.sg>
Subject RE: Failed to install Hadoop on WinXP
Date Thu, 28 Jan 2010 23:42:09 GMT

Good Day,

My Hadoop version is 0.19.1. I have successfully configured to run it on a Windows machine.

Here is the configuration that I performed:

1. I put the Hadoop files under this folder C:\cygwin\usr\local\hadoop

2. Below is the hadoop-site.xml that I use.

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/cygwin/tmp/hadoop-${user.name}</value>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
<property>
<name>mapred.job.tracker</name>
<value>localhost:9001</value>
</property>
<property>
<name>mapred.job.reuse.jvm.num.tasks</name>
<value>-1</value>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
<property>
<name>webinterface.private.actions</name>
<value>true</value>
</property>
</configuration>

-----Original Message-----
From: Yang Li [mailto:liyang@cn.ibm.com]
Sent: Thu 1/28/2010 9:41 PM
To: common-user@hadoop.apache.org
Subject: Re: Failed to install Hadoop on WinXP
 
I met the same problem on WinXP+Cygwin and fixed it by
either:
- moving to a linux box (VMWare works very well)
or:
- configuring a "mapred.child.tmp" parameter in core-site.xml

I cannot explain why and how "mapred.child.tmp" is related to the problem. 
>From source code, it seems to be a JVM issue on windows. Hope this helps.

Best regards,
-----------------------
Li Yang, ext. 22056, User Technologies Development, Shanghai, China




Yura Taras <yura.taras@gmail.com> 
2010-01-28 00:41
Please respond to
common-user@hadoop.apache.org


To
common-user@hadoop.apache.org
cc

Subject
Failed to install Hadoop on WinXP



Hi all
I'm trying to deploy pseudo-distributed cluster on my devbox which
runs under WinXP. I did following steps:
1. Installed cygwin with ssh, configured ssh
2. Downloaded hadoop and extracted it, set JAVA_HOME and HADOOP_HOME
env vars (I made a symlink to java home, so it don't contain spaces)
3. Adjusted conf/hadoop-env.sh to point to correct JAVA_HOME
4. Adjusted conf files to following values:
   * core-site.xml:
<configuration>
    <property>
      <name>hadoop.tmp.dir</name>
      <value>/hdfs/hadoop</value>
      <description>A base for other temporary directories.</description>
    </property>
    <property>
        <name>fs.default.name</name>
        <value>hdfs://localhost:8888</value>
    </property>
</configuration>

  * hdfs-site.xml:
<configuration>
  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
</configuration>

  * mapred-site.xml:
<configuration>
  <property>
    <name>mapred.job.tracker</name>
    <value>localhost:9999</value>
  </property>
</configuration>

5. Next I execute following line: $ bin/hadoop namenode -format &&
bin/start-all.sh && bin/hadoop fs -put conf input && bin/hadoop jar
hadoop-0.20.1-examples.jar grep input output 'dfs[a-z.]'

I receive following exception:
localhost: starting tasktracker, logging to
/home/ytaras/hadoop/bin/../logs/hadoop-ytaras-tasktracker-bueno.out
10/01/27 18:23:55 INFO mapred.FileInputFormat: Total input paths to 
process : 13
10/01/27 18:23:56 INFO mapred.JobClient: Running job: 
job_201001271823_0001
10/01/27 18:23:57 INFO mapred.JobClient:  map 0% reduce 0%
10/01/27 18:24:09 INFO mapred.JobClient: Task Id :
attempt_201001271823_0001_m_000014_0, Status : FAILED
java.io.FileNotFoundException: File
D:/hdfs/hadoop/mapred/local/taskTracker/jobcache/job_201001271823_0001/attempt_201001271823_0001_m_000014_0/work/tmp
does not exist.
        at 
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361)
        at 
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
        at 
org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519)
        at org.apache.hadoop.mapred.Child.main(Child.java:155)

!! SKIP - above exception few times !!

10/01/27 18:24:51 INFO mapred.JobClient: Job complete: 
job_201001271823_0001
10/01/27 18:24:51 INFO mapred.JobClient: Counters: 0
java.io.IOException: Job failed!
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
        at org.apache.hadoop.examples.Grep.run(Grep.java:69)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.examples.Grep.main(Grep.java:93)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at 
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at 
org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at 
org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)



Am I doing something wrong (don't say just 'use Linux' :-) )?
Thanks



Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message