giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From XingFENG <>
Subject Re: When do Giraph vertices receive their messages?
Date Mon, 10 Nov 2014 10:49:14 GMT
Hi Vincentius Martin,

Since Giraph is based on Pregel, I would refer you to the paper *Pregel: A
System for Large-Scale Graph Processing *for more details.

Briefly speaking, in each superstep,
1. a worker (which is responsible for a partition of vertices) receives
messages from others. A worker then divided these messages according to the
destID and active vertices which have incoming messages.
2. a worker runs *compute* function of each active vertex. Meanwhile, the
*compute* function may generate messages to other vertices. These messages
are buffered, combined and sent in batches in an asynchronous way.
3. after a worker finishes *compute* function of all active vertex, it
waits for all other workers finishing their *compute* functions. What is
more, it waits for all sending tasks to finish to ensure all messages can
be received in next superstep. Then every worker goes into next superstep.

For your second problem, messages are stored in a buffer.

On Mon, Nov 10, 2014 at 6:14 PM, Puneet Agarwal <> wrote:

> These are some very interesting questions. I also would like to know the
> answers to these.
> - Puneet
> IIT Delhi, India
>   On Monday, November 10, 2014 9:30 AM, Vincentius Martin <
>> wrote:
> I am curious about how does Giraph receive messages before processing it
> I know that they use their accepted messages in the compute() method on
> the next superstep, but when do they receive it? If it is before the
> checkpoint process, is there any part in the documentation/code that I can
> see to understand it?
> Also, what mechanism that Giraph use to store messages before superstep
> S+1? Are they store it in a buffer or disk first?
> I still cannot find anything about this.
> Regards,
> Vincentius Martin

Best Regards.
PhD Candidate
Database Research Group

School of Computer Science and Engineering
University of New South Wales
NSW 2052, Sydney

Phone: (+61) 413 857 288

View raw message