hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <tdunn...@veoh.com>
Subject Re: submitting map-reduce jobs without creating jar file ?
Date Tue, 22 Apr 2008 22:31:24 GMT


Grool might help you.

It allows you to write full on java-based map-reduce programs using a very
simple scripting interface.  It has a jar with all of the code needed, but
your scripts wouldn't have to be packaged into anything.  As a scripting
language, it is very easy to integrate and the map-reduce jobs are easy to
manipulate.

If you already have code to perform the manipulations, then you can still
use grool to launch customized jobs since grool is based on groovy and can
call java ad libitem.


On 4/22/08 3:19 PM, "Tarandeep Singh" <tarandeep@gmail.com> wrote:

> hi,
> 
> Can I submit a map-reduce job without creating the jar file (and using
> $HADOOP_HOME/bin/hadoop script). I looked into the hadoop script and
> it is invoking org.apache.hadoop.util.RunJar class. Should I (or
> rather do I) have to do the same thing as this class is doing if I
> don't want to use the conventional job scheduling method.
> 
> I have this requirement that my application receive information from
> external source that would govern the behavior of map-reduce classes
> (so I have to create them on the fly or at least customize them)
> 
> Any pointers will be useful...
> 
> thanks,
> Taran


Mime
View raw message