hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bejoy KS" <bejoy.had...@gmail.com>
Subject Re: Streaming in mapreduce
Date Sat, 16 Jun 2012 09:30:53 GMT
Hi Pedro

In simple terms Streaming API is used in hadoop if you have your mapper or reducer is in any
language other than java . Say ruby or python. 


Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Pedro Costa <psdc1978@gmail.com>
Date: Sat, 16 Jun 2012 10:23:20 
To: mapreduce-user@hadoop.apache.org<mapreduce-user@hadoop.apache.org>
Reply-To: mapreduce-user@hadoop.apache.org
Subject: Re: Streaming in mapreduce

I still don't get why hadoop streaming is useful. If I have man and reduce functions defined
in shell script, like the one below, why should I use Hadoop?
cat someInputFile | shellMapper.sh | shellReducer.sh > someOutputFile


On 16/06/2012, at 01:21, Ruslan Al-Fakikh <metaruslan@gmail.com> wrote:

> Hi Pedro,
> 
> You can find it here
> http://wiki.apache.org/hadoop/HadoopStreaming
> 
> Thanks
> 
> On Sat, Jun 16, 2012 at 2:46 AM, Pedro Costa <psdc1978@gmail.com> wrote:
>> Hi,
>> 
>> Hadoop mapreduce can be used for streaming. But what is streaming from the point
of view of mapreduce? For me, streaming are video and audio data.
>> 
>>  Why mapreduce supports streaming?
>> 
>> Can anyone give me an example on why to use streaming in mapreduce?
>> 
>> Thanks,
>> Pedro

Mime
View raw message