giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maja Kabiljo <>
Subject Re: shared aggregators
Date Wed, 29 May 2013 17:19:46 GMT

First, the total number of messages is the same, since worker won't be
sending aggregators to itself, so we have K + (N-1)*K.

If master would be sending aggregators to all workers, it would have to
send K*N amount of data. This way master only sends K, and then as Greg
said all workers work in parallel - each sending additional K (or more
precisely K/N * (N-1)).

Does this make it clearer?


On 5/29/13 9:22 AM, "Greg Malewicz" <> wrote:

>Consider parallel time.
>On 5/29/13 9:08 AM, "Claudio Martella" <> wrote:
>>I have a question about the design of shared aggregators.
>>Documentation says:
>>"After MasterCompute.compute, master doesn't do the distribution of all
>>aggregators to all workers, but aggregators again have their owners.
>>only sends each aggregator to its owner, and then each worker distributes
>>the aggregators which it owns to all other workers."
>>Why are the aggregator values not sent directly by the master to all
>>workers, instead of doing the two hops it does now?
>>Suppose I have K aggregators and N workers, the current design requires K
>>K*N messages.
>>If the master would send the aggregators values to all the workers
>>directly, we would have K*N messages, or actually N messages with K
>>Am I missing something?
>>   Claudio Martella

View raw message