giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <>
Subject Inter- and intra-node message passing
Date Mon, 04 Feb 2013 17:48:34 GMT
Giraph runs on a Hadoop cluster as a map reduce job. Each worker is hence a
different task and it runs as a different jvm. This means that you can have
two workers running in the same machine. Hence it can happen that two
vertices running on two different workers executed in the same machine
might exchange messages over the network. The practical impact depends on
the host operating system.
On the other side, two vertices residing on the same worker but being
computed by different compute threads will behave as mentioned in the
previous email. If you would like to minimize the first behavior, you
should ensure a single worker is executed on each machine, and set
adequately the number of compute threads per worker.

Hope this helps,

On Monday, February 4, 2013, Daglis Alexandros wrote:

>  Hello Claudio,
> Thank you for your prompt answer!
> So, vertices that belong to the same worker thread do not require access
> to the network in order to exchange messages.
> However, what about *different worker threads* that reside on different
> cores *of the same node*?
> Cheers,
> Alexandros
>  ------------------------------
> *From:* Claudio Martella []
> *Sent:* Monday, February 04, 2013 6:19 PM
> *To:*
> *Subject:* Re: Inter- and intra-node message passing
>   Hi Alexandros,
>  if two vertices are on the same worker, the message does not pass
> through the network but it is put directly in the mailbox of the
> destination vertex.
>  Cheers,
> Claudio
> On Mon, Feb 4, 2013 at 6:03 PM, Alexandros Daglis <
>> wrote:
>>  Hello everybody,
>> I was wondering about the message-passing protocol: is there a difference
>> if two communicating threads are on the same node, as opposed to being on
>> different ones? Is communication achieved through memory whenever the
>> threads are local to the node, or does it always default to the network?
>> I tried answering that question by going through the code, but I haven't
>> seen any high-level difference in handling those two different cases. I
>> would appreciate if someone could give me a hint on that.
>> Thank you in advance.
>> Alexandros
>  --
>    Claudio Martella

   Claudio Martella

View raw message