hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravi Phulari <rphul...@gmail.com>
Subject Re: Running a MapReduce Program
Date Sun, 14 Jun 2009 05:37:41 GMT
Assuming your Hadoop setup ( local-standalone, pseudo-distributed or
fully-distributed) is done .
If not please follow
http://hadoop.apache.org/core/docs/current/quickstart.html
Next - 
Write Example Word count program  in WordCount.java -
http://hadoop.apache.org/core/docs/current/mapred_tutorial.html#Example%3A+W
ordCount+v1.0    

Finally 
Assuming HADOOP_HOME is the root of the installation and HADOOP_VERSION is
the Hadoop version installed, compile WordCount.java and create a jar:

$ mkdir wordcount_classes
$ javac -classpath ${HADOOP_HOME}/hadoop-${HADOOP_VERSION}-core.jar -d
wordcount_classes WordCount.java
$ jar -cvf /usr/joe/wordcount.jar -C wordcount_classes/ .

Assuming that:

/usr/joe/wordcount/input - input directory in HDFS
/usr/joe/wordcount/output - output directory in HDFS
Sample text-files as input:
$ bin/hadoop dfs -ls /usr/joe/wordcount/input/
/usr/joe/wordcount/input/file01
/usr/joe/wordcount/input/file02

$ bin/hadoop dfs -cat /usr/joe/wordcount/input/file01
Hello World Bye World

$ bin/hadoop dfs -cat /usr/joe/wordcount/input/file02
Hello Hadoop Goodbye Hadoop

Run the application:

$ bin/hadoop jar /usr/joe/wordcount.jar org.myorg.WordCount
/usr/joe/wordcount/input /usr/joe/wordcount/output


I hope this will help you compiling and running your WordCount program .

If you any further problems in running this example then post steps you have
followed and stack error out put .

-
Ravi  

On 6/13/09 8:50 PM, "bharath vissapragada" <bharat_v@students.iiit.ac.in>
wrote:

> The details provided in this page are not that clear . I am a newbie to java
> , so please take the pain of explaining because .. i am not able to compile
> the java code to make a jar file . It is giving many errors such as
> "packages not found" . This means that there is some problem with the path ,
> Classpath etc. Someone kindly explain
> 
> Thanks
> 
> On Sun, Jun 14, 2009 at 1:36 AM, Ravi Phulari <rphulari@yahoo-inc.com>wrote:
> 
>> > You can find answers to your all questions on this web page .
>> >
>> > http://hadoop.apache.org/core/docs/current/mapred_tutorial.html
>> >
>> > -
>> > Ravi
>> >
>> > On 6/13/09 3:30 AM, "bharath vissapragada" <bharat_v@students.iiit.ac.in>
>> > wrote:
>> >
>> > Also please tell me , how to set path and stuff , and directory in wch the
>> > code mst be compiled etc,
>> > Thanks
>> >
>> > On Sat, Jun 13, 2009 at 3:58 PM, bharath vissapragada <
>> > bharat_v@students.iiit.ac.in> wrote:
>> >
>>> > > Hi all,
>>> > > I wrote a java code(map-reduce). Can anyone tell me in detail, how
to
run
>>> > > it on hadoop (right from how to create a jar file).. or send me a link
>>> > > specifying the same.
>>> > >
>>> > > Thanks in advance
>>> > >
>>> > >
>> >
>> >
-- 



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message