giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maja Kabiljo <majakabi...@fb.com>
Subject Re: shared aggregators
Date Wed, 29 May 2013 17:19:46 GMT
Claudio,

First, the total number of messages is the same, since worker won't be
sending aggregators to itself, so we have K + (N-1)*K.

If master would be sending aggregators to all workers, it would have to
send K*N amount of data. This way master only sends K, and then as Greg
said all workers work in parallel - each sending additional K (or more
precisely K/N * (N-1)).

Does this make it clearer?

Maja

On 5/29/13 9:22 AM, "Greg Malewicz" <malewicz@fb.com> wrote:

>Consider parallel time.
>
>Greg
>
>On 5/29/13 9:08 AM, "Claudio Martella" <claudio.martella@gmail.com> wrote:
>
>>Hi,
>>
>>I have a question about the design of shared aggregators.
>>Documentation says:
>>
>>"After MasterCompute.compute, master doesn't do the distribution of all
>>aggregators to all workers, but aggregators again have their owners.
>>Master
>>only sends each aggregator to its owner, and then each worker distributes
>>the aggregators which it owns to all other workers."
>>
>>Why are the aggregator values not sent directly by the master to all
>>workers, instead of doing the two hops it does now?
>>
>>Suppose I have K aggregators and N workers, the current design requires K
>>+
>>K*N messages.
>>If the master would send the aggregators values to all the workers
>>directly, we would have K*N messages, or actually N messages with K
>>values
>>each.
>>
>>Am I missing something?
>>
>>Best,
>>Claudio
>>
>>-- 
>>   Claudio Martella
>>   claudio.martella@gmail.com
>


Mime
View raw message