hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guang-Nan Cheng <chen...@gmail.com>
Subject Re: Passing files and directory structures to the map reduce cluster via hadoop streaming?
Date Wed, 29 Jun 2011 06:44:32 GMT
Well, my bad. I made a simple test and confirmed that  -files works that way
already.

For the two guys that "answered" my question, sorry I asked the question
unclearly... I don't see how those two projects related to the question,
 but thank you. :D




On Wed, Jun 29, 2011 at 12:35 AM, Abhinay Mehta <abhinay.mehta@gmail.com>wrote:

> We use Mandy: https://github.com/forward/mandy for this.
>
>
> On 28 June 2011 17:26, Nick Jones <nick.jones@amd.com> wrote:
>
> > Take a look at Wukong from the guys at Infochimps:
> > https://github.com/mrflip/**wukong <https://github.com/mrflip/wukong>
> >
> >
> > On 06/28/2011 11:19 AM, Guang-Nan Cheng wrote:
> >
> >> I'm fancied about passing a whole ruby app to streaming, so I don't need
> >> to
> >> bother with ruby file dependencies.
> >>
> >> For example,
> >>
> >> ./streaming
> >>
> >> ...
> >> -mapper 'ruby aaa/bbb/ccc'
> >> -files  aaa<--- pass the folder
> >>
> >>
> >>
> >>
> >> Is this supported already? If not, any tips on how to make this work?
> I'm
> >> willing to add some code by myself and rebuild the streaming jar.
> >>
> >
> > --
> > Nick Jones
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message