hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans" <jdcry...@apache.org>
Subject Re: Running map/reduce written in Ruby on Hbase
Date Fri, 19 Sep 2008 01:29:34 GMT
Hui Ding,

This wouldn't work very well. Streaming is defined so that you pass programs
(any) that can take in input and an output in the filesystem, not HBase
tables. You should instead try to use JRuby like we do for the shell. It
requires some more setup, but since it all runs inside the JVM it eventually
works.

I see that more and more users are interested in using JRuby/Jython for MR
jobs and I know that some companies already uses a wrapper for that ("Happy"
anyone?). I'm sure many would be insterested in seeing this kind of work.

J-D

On Thu, Sep 18, 2008 at 7:57 PM, Ding, Hui <hui.ding@sap.com> wrote:

> Hi all,
>
> I wanted to run some map/reduce job but I'd like to do that in Ruby, is
> this possible with Hadoop Streaming?
> My understanding is that I will provide mapper/reducer in Ruby and
> supply that to Hadoop Streamining, and since hbase can be a source/sink
> of map/reduce, I should be able to access the tables, right?
>
> And as far as setup is concered, I just need to have a ruby interpreter
> set up on each of the machine in the cluster?
>
> Thanks a lot!
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message