Mailing-List: contact hadoop-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-user@lucene.apache.org
Received-SPF: pass (herse.apache.org: domain of roger.edchen@gmail.com
 designates 209.85.146.183 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=beta;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
        b=dw0bNZwa7aEt+iY3nbnn/mMsLG+6pydPWj4DVz4W8/0mQC15UaUy+argjnulxe2kJ773y/2Pt8gUkxqOukaXPG3npETZZ1aA3RGkHQ5Qe2+Pg7nLESv39rkL2dAaCj8jiy2d7xrb4EhQlgZ7Zk53UV3x/1zi9+ZIFVfU+QVgZhY=
Message-ID: <ce8557b50706020527t7964a22aw2a2aefb69e6bbcf1@mail.gmail.com>
Date: Sat, 2 Jun 2007 08:27:27 -0400
From: "Erdong (Roger) CHEN" <roger.edchen@gmail.com>
To: hadoop-user@lucene.apache.org
Subject: [Hadoop DFS] hadoop does not see my input file
In-Reply-To: <ce8557b50706012110v319359ebp40e7c630dc9c021e@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <ce8557b50706012110v319359ebp40e7c630dc9c021e@mail.gmail.com>

Hi all,

Could anyone help me to figure out why hadoop does not see my input file?

I have three computers rosetta8, rosetta9,and rosetta10. rosetta8 is
listed in masters, rosetta9 and rosetta10 are listed in slaves. I run
bin/start-dfs.sh and bin/start-mapred.sh on rosetta8. This is my
hapood-site.xml. I am pretty sure that I followed the installation and
configuration online and the folder /tmp/in-dir/ is not empty.

I tried the following two commands:
./bin/hadoop dfs -ls /tmp/in-dir/
Found 0 items
./bin/hadoop dfs -ls /tmp/
Found 0 items

I tried both two settings for mapred.job.tracker, local and
rosetta8:50034. Both don't work.

<property>
  <name>mapred.job.tracker</name>
  <value>local</value>
  <value>rosetta8:50034</value>
</property>

<property>
  <name>fs.default.name</name>
  <value>rosetta8:50033</value>
</property>

Command that I run:
./bin/hadoop jar hadoop-0.12.3-examples.jar wordcount -m 3 -r 2
/tmp/in-dir/ /tmp/out-dir/

Error message that I get:
org.apache.hadoop.mapred.InvalidInputException: Input path doesnt
exist : /tmp/in-dir
        at org.apache.hadoop.mapred.InputFormatBase.validateInput(InputFormatBase.java:138)
        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:326)
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:543)
        at org.apache.hadoop.examples.WordCount.main(WordCount.java:148)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:585)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:143)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:40)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:585)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
edc@rosetta8:~/hadoop-install/hadoop$ ./bin/hadoop jar
hadoop-0.12.3-examples.jar wordcount -m 3 -r
2/afs/csail.mit.edu/u/e/edc/hadoop-install/hadoop/in-dir/
/tmp/out-dir/
ERROR: Integer expected instead of
2/afs/csail.mit.edu/u/e/edc/hadoop-install/hadoop/in-dir/
wordcount [-m <maps>] [-r <reduces>] <input> <output>
edc@rosetta8:~/hadoop-install/hadoop$ ./bin/hadoop jar
hadoop-0.12.3-examples.jar wordcount -m 3 -r 2
/afs/csail.mit.edu/u/e/edc/hadoop-install/hadoop/in-dir/ /tmp/out-dir/
org.apache.hadoop.mapred.InvalidInputException: Input path doesnt
exist : /afs/csail.mit.edu/u/e/edc/hadoop-install/hadoop/in-dir
        at org.apache.hadoop.mapred.InputFormatBase.validateInput(InputFormatBase.java:138)
        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:326)
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:543)
        at org.apache.hadoop.examples.WordCount.main(WordCount.java:148)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:585)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:143)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:40)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:585)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:155)


Erdong Chen