hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amareshwari Sri Ramadasu <amar...@yahoo-inc.com>
Subject Re: Hadoop Pipes Error
Date Thu, 31 Mar 2011 10:59:06 GMT

The inputformat is present in test jar. So, pass -libjars <full path to testjar> to
your command. libjars option should be passed before program specific options. So, it should
be just after your -D parameters.


On 3/31/11 3:45 PM, "Adarsh Sharma" <adarsh.sharma@orkash.com> wrote:

Amareshwari Sri Ramadasu wrote:
Re: Hadoop Pipes Error You can not run it with TextInputFormat. You should run it with org.apache.hadoop.mapred.pipes
.WordCountInputFormat. You can pass the input format by passing it in -inputformat option.
I did not try it myself, but it should work.

Here is the command that I am trying and it results in exception:

bash-3.2$ bin/hadoop pipes -D hadoop.pipes.java.recordreader=true -D hadoop.pipes.java.recordwriter=true
 -inputformat org.apache.hadoop.mapred.pipes.WordCountInputFormat -input gutenberg -output
gutenberg-out101 -program bin/wordcount-nopipe
Exception in thread "main" java.lang.ClassNotFoundException: org.apache.hadoop.mapred.pipes.WordCountInputFormat
        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:247)
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
        at org.apache.hadoop.mapred.pipes.Submitter.getClass(Submitter.java:372)
        at org.apache.hadoop.mapred.pipes.Submitter.run(Submitter.java:421)
        at org.apache.hadoop.mapred.pipes.Submitter.main(Submitter.java:494)

Thanks , Adarsh

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message