incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Brickley <>
Subject Re: Port to YARN: GIRAPH and HAMA
Date Tue, 13 Sep 2011 18:20:58 GMT
On 13 September 2011 19:47, Avery Ching <> wrote:

> Perhaps more practically, I wonder if it would be possible for someone from
> the Hama team to refactor our code a bit to support Hama-style BSP in
> Giraph?  Certainly would be a pretty cool project...

Maybe this is crazy, but: I was wondering...  Pregel's basic API
approach is pretty straightforward, gloriously simple even. Could we
have platform-neutral APIs that allowed portability of applications
between  Pregel-based platforms? At least for Java...

Right now, those of us who are more 'application people' than platform
developers, are left searching around on 'pregel opensource' and have
to try to guess which of the various Pregel-eseque platforms is
looking most healthy. For example, my summer vacation project was
checking out GoldenOrbOS. Yet by the time I get back, the Mahout list
was buzzing with discussion of Giraph, so I took a look at that (and
was pleasantly suprised).

There is clearly a lot of energy and creativity right now going into
this kind of distributed graph processing platform. That suggests to
me that *finalising* cross-platform APIs would be premature. But it is
also a time when platforms have a certain amount of flexibility that
they will loose as they get adopted and embedded within products and
processes. Could a Pregel-like Java API be agreed between platforms
(e.g. let's consider Giraph, Hama, GoldenOrbOS), so that those of us
investigating applications could proceed with some hope of later
portability. This might be cheaper than trying to persuade Giraph to
rebuild on top of Hama, or suchlike. Anyone care to make a first pass
at suggesting some common interfaces?



View raw message