hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White" <tom.e.wh...@gmail.com>
Subject Re: JavaSpaces (Blitz?) and hadoop - comparison?
Date Thu, 01 Mar 2007 13:50:41 GMT
Good question. JavaSpaces is actually very general, so I would ask how
the Replicated Worker Pattern, which is fits nicely with JavaSpaces
(http://today.java.net/pub/a/today/2005/04/21/farm.html,
https://computeserver.dev.java.net/) compares with Hadoop and
MapReduce.

My take is that at a high level JavaSpaces RWP is good for
distributing jobs that don't operate on large datasets whereas Hadoop
(MapReduce) is good for operating on very large datasets. JavaSpaces
doesn't really have mechanisms for distributing large quantities of
data in the way that HDFS does. On the other hand, JavaSpaces is good
for sharing modest sized data objects - with MapReduce you are sharing
data, so you have to think carefully how to encode the data that the
map and reduce tasks operate on. JavaSpaces in general allows you a
richer computational model (compared to MapReduce) - but this
generality comes at the price of being able to perform well for
certain classes of application.

Put another way: JavaSpaces RWP is a good fit for writing a program to
calculate if a large number is prime (since the subtasks don't need to
use much intermediate data), whereas Hadoop MapReduce is a good fit
for counting web server access hits by host (since the object is to
analyse a large set of data).

(I think they're both great pieces of technology BTW.)

Tom

On 01/03/07, Dan Creswell <dan.creswell@lonecrusader.co.uk> wrote:
> Hi,
>
> I'm the author of Blitz as it happens :)
>
> Blitz has various different modes of operation.  It can be persistent
> but also can operate in "memory-only" configuration.
>
> The basic difference would be that Blitz is just a core element around
> which you could build a MapReduce implementation (in fact I have done in
> the past) etc. Hadoop by contrast has a core of it's own based on a GFS
> equivalent and also includes the framework for MapReduce etc.
>
> In conclusion, Blitz provides the base of a stack and Hadoop has that
> base (in another form) plus additional layers (MapReduce etc).
>
> Hope that helps,
>
> Dan.
>
> Tomi N/A wrote:
> > I just came across a technology which sounded interesting, but doesn't
> > seem very wide spread called JavaSpaces. An implementation is Blitz
> > (http://www.dancres.org/blitz/).
> >
> >> From what I see, it seems to be a distributed computation and
> > persistence engine so it made me wonder if anyone on this list would
> > know anything about it and, maybe, compare the two technologies. Well?
> > :)
> >
> > Cheers,
> > t.n.a.
> >
>
>

Mime
View raw message