cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brian O'Neill" <b...@alumni.brown.edu>
Subject MapReduce on Cassandra using Ruby and REST!
Date Thu, 01 Dec 2011 19:01:47 GMT
I know I've been spamming the list a bit with new features for Virgil, but
this one is actually really cool...

Enamored with what Riak provides as far as map/reduce via HTTP,
http://wiki.basho.com/MapReduce.html#MapReduce-via-the-HTTP-API

We implemented the same thing for Virgil/Cassandra.   Simply write a script
in ruby, then POST that script to Virgil.  Virgil will kick off the Hadoop
job against Cassandra using column families for input and output.

Right now it only supports Ruby, but there is nothing preventing us from
adding support for other languages.  We'll probably throw Groovy in there
as well.
Check it out when you get a chance...
http://brianoneill.blogspot.com/2011/12/hadoopmapreduce-on-cassandra-using-ruby.html

After we add the ability to push the job to an existing Hadoop cluster,
we'll move on to PIG support and CRUD operations in the GUI.

stay tuned,
-brian

-- 
Brian ONeill
Lead Architect, Health Market Science (http://healthmarketscience.com)
mobile:215.588.6024
blog: http://weblogs.java.net/blog/boneill42/
blog: http://brianoneill.blogspot.com/

Mime
View raw message