giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <>
Subject Re: Dynamic Graphs
Date Wed, 21 Aug 2013 13:30:36 GMT
As I said, the injection of the new vertices/edges would have to be done
"manually", hence without any support of the infrastructure. I'd suggest
you implement a WorkerContext class that supports the reading of a specific
file with a specific format (under your control) from HDFS, and that is
accessed by this particular "special" vertex (e.g. based on the vertex ID).

Does this make sense?

On Wed, Aug 21, 2013 at 2:13 PM, Marco Aurelio Barbosa Fagnani Lotz <> wrote:

>  Dear Mr. Martella,
> Once achieved the conditions for updating the vertex data base, what it
> the best way for the Injector Vertex to call an input reader again?
> I am able to access all the HDFS data, but I guess the vertex would need
> to have access to the input splits and also the vertex input format that I
> designate. Am I correct? Or there is a way that one can just ask Zookeeper
> to create new splits and distribute to the workers from given a path in DFS?
> Best Regards,
> Marco Lotz
>  ------------------------------
> *From:* Claudio Martella <>
> *Sent:* 14 August 2013 15:25
> *To:*
> *Subject:* Re: Dynamic Graphs
>  Hi Marco,
>  Giraph currently does not support that. One way of doing this would be
> by having a specific (pseudo-)vertex to act as the "injector" of the new
> vertices and edges For example, it would read a file from HDFS and call the
> mutable API during the computation, superstep after superstep.
> On Wed, Aug 14, 2013 at 3:02 PM, Marco Aurelio Barbosa Fagnani Lotz <
>> wrote:
>>  Hello all,
>> I would like to know if there is any form to use dynamic graphs with
>> Giraph. By dynamic one can read graphs that may change while Giraph is
>> computing/deliberating. The changes are in the input file and are not
>> caused by the graph computation itself.
>> Is there any way to analyse it using Giraph? If not, anyone has any
>> idea/suggestion if it is possible to modify the framework in order to
>> process it?
>> Best Regards,
>> Marco Lotz
>  --
>    Claudio Martella

   Claudio Martella

View raw message