hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abhinay Mehta <abhinay.me...@gmail.com>
Subject Re: Passing files and directory structures to the map reduce cluster via hadoop streaming?
Date Tue, 28 Jun 2011 16:35:37 GMT
We use Mandy: https://github.com/forward/mandy for this.


On 28 June 2011 17:26, Nick Jones <nick.jones@amd.com> wrote:

> Take a look at Wukong from the guys at Infochimps:
> https://github.com/mrflip/**wukong <https://github.com/mrflip/wukong>
>
>
> On 06/28/2011 11:19 AM, Guang-Nan Cheng wrote:
>
>> I'm fancied about passing a whole ruby app to streaming, so I don't need
>> to
>> bother with ruby file dependencies.
>>
>> For example,
>>
>> ./streaming
>>
>> ...
>> -mapper 'ruby aaa/bbb/ccc'
>> -files  aaa<--- pass the folder
>>
>>
>>
>>
>> Is this supported already? If not, any tips on how to make this work? I'm
>> willing to add some code by myself and rebuild the streaming jar.
>>
>
> --
> Nick Jones
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message