Mailing-List: contact giraph-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: giraph-user@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of janlugt@gmail.com designates
 209.85.215.175 as permitted sender)
MIME-Version: 1.0
Date: Tue, 24 Apr 2012 08:53:42 -0700
Message-ID: 
 <CAMOYj4B3bt_vTXjmH3yDT7uzuBzm-ygXyatCSzX46tA6SrMDJQ@mail.gmail.com>
Subject: Missing features in Giraph
From: Jan van der Lugt <janlugt@gmail.com>
To: giraph-user@incubator.apache.org
Content-Type: multipart/alternative; boundary=90e6ba5bb86f5204b104be6ec27e

--90e6ba5bb86f5204b104be6ec27e
Content-Type: text/plain; charset=ISO-8859-1

Dear all,

After having worked with Giraph for some weeks I feel like there are two
features 'missing' in Giraph. It may be I simply missed them in the
Javadoc, since the documentation is a work in progress at this point. In
another Google Pregel-clone, Stanford GPS, it is possible to define a
global object map, which can be used by all workers to share data, like the
current phase in the algorithm. I have not been able to find such a feature
in Giraph. Of course it would be possible to (ab)use aggregators for this,
but I doubt this is the easiest or most efficient approach. Furthermore, it
would be very helpful if there would be one special vertex that has the
role of a master. This should not have to correspond to an existing vertex
in the graph, it would be easier if it were not, actually. This master node
would then be able to perform some centralized steps in the algorithm, of
which the output can then be shared with other workers via the global
object map. The master node could have the same interface as the workers
(compute(), getAggregator(), getConf(), etc.). Again, it would be possible
to solve this otherwise, for example in the VertexReader, but this would
make code less elegant and would require picking a vertex id that does not
exist in the graph, which is difficult if the input is not known in advance.

I realize I am biased because my earlier experiences with Stanford GPS, but
I feel these features will not be very difficult to implement or would add
bulkiness to the API. They can make the implementation of many graph
algorithms easier, though, because many of these algorithms have some
notion of a centralized master node. During the next 5 months I will be
working with Giraph for my Master's project, so I would be more than
willing to help out implementing these features, ideally after receiving
some pointers from more experienced Giraph developers.

Regards,
Jan van der Lugt

--90e6ba5bb86f5204b104be6ec27e
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div class=3D"gmail_extra">Dear all,</div><div class=3D"gmail_extra"><br></=
div><div class=3D"gmail_extra">After having worked with Giraph for some wee=
ks I feel like there are two features &#39;missing&#39; in Giraph. It may b=
e I simply missed them in the Javadoc, since the documentation is a work in=
 progress at this point.=A0In another Google Pregel-clone, Stanford GPS, it=
 is possible to define a global object map, which can be used by all worker=
s to share data, like the current phase in the algorithm. I have not been a=
ble to find such a feature in Giraph. Of course it would be possible to (ab=
)use aggregators for this, but I doubt this is the easiest or most efficien=
t approach. Furthermore, it would be very helpful if there would be one spe=
cial vertex that has the role of a master. This should not have to correspo=
nd to an existing vertex in the graph, it would be easier if it were not, a=
ctually. This master node would then be able to perform some centralized st=
eps in the algorithm, of which the output can then be shared with other wor=
kers via the global object map. The master node could have the same interfa=
ce as the workers (compute(), getAggregator(), getConf(), etc.). Again, it =
would be possible to solve this otherwise, for example in the VertexReader,=
 but this would make code less elegant and would require picking a vertex i=
d that does not exist in the graph, which is difficult if the input is not =
known in advance.</div>
<div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">I realize I=
 am biased because my earlier experiences with Stanford GPS, but I feel the=
se features will not be very difficult to implement or would add bulkiness =
to the API. They can make the implementation of many graph algorithms easie=
r, though, because many of these algorithms have some notion of a centraliz=
ed master node. During the next 5 months I will be working with Giraph for =
my Master&#39;s project, so I would be more than willing to help out implem=
enting these features, ideally after receiving some pointers from more expe=
rienced Giraph developers.</div>
<div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">Regards,</d=
iv><div class=3D"gmail_extra">Jan van der Lugt</div>

--90e6ba5bb86f5204b104be6ec27e--