hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daryn Sharp <da...@yahoo-inc.com>
Subject Re: Hadoop MapReduce
Date Tue, 23 Apr 2013 17:47:15 GMT
MR has a "local mode" that does what you want.  Pig has the ability to use this mode.  I did
a quick search but didn't immediately find a good link to documentation, but hopefully this
gets you going in the right direction.

Daryn

On Apr 22, 2013, at 6:01 PM, David Gkogkritsiani wrote:

Helllo,

I have undertaken my diploma thesis on Hadoop MapReduce and I have been requested to I do
an application written in MapReduce.
I found on internet this code and I ran the code :

http://paste.ubuntu.com/5591999/

How can I add the code to stores the pages somewhere locally (text only, not Images) and then
have to be processed . ie,I should a Mapreduce code, which would download pages from the web
and store on the local file system and not the HDFS.
After ,I run the quest (program) in order to not depend on network speed.

Because ,my network is so slow.

I do this to improvement performance.

I am running Hadoop Version 0.20.2 .
I am new to Hadoop and am kinda lost and any help would be greatly appreciated.

Thanks in advance for any assistance !


Mime
View raw message