Mailing-List: contact dev-help@giraph.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@giraph.apache.org
Received-SPF: unknown (nike.apache.org: error in processing during lookup of
 prvs=986124b3a8=majakabiljo@fb.com)
From: Maja Kabiljo <majakabiljo@fb.com>
To: "dev@giraph.apache.org" <dev@giraph.apache.org>
Subject: Re: shared aggregators
Thread-Topic: shared aggregators
Thread-Index: AQHOXIhAsDl3NJTL4UahbaxIZa31IZkczc2A//+aj4CAAJGSAP//jvMA
Date: Wed, 29 May 2013 19:16:09 +0000
Message-ID: 
 <1F592C080E9ACB4CB1C9EA1865BF3EFA0D1A0801@PRN-MBX01-2.TheFacebook.com>
In-Reply-To: 
 <CAFJOoJeaxmvMb+05+xo6sY_YqL5E2jfa3kRZtUekXetfOrDn6w@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <E9C48BA38452044FB8E3FF27C017E675@fb.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Feel free to expand the explanation on the website :-)

On 5/29/13 12:00 PM, "Claudio Martella" <claudio.martella@gmail.com> wrote:

>Yes totally. Thanks.
>
>
>On Wed, May 29, 2013 at 7:19 PM, Maja Kabiljo <majakabiljo@fb.com> wrote:
>
>> Claudio,
>>
>> First, the total number of messages is the same, since worker won't be
>> sending aggregators to itself, so we have K + (N-1)*K.
>>
>> If master would be sending aggregators to all workers, it would have to
>> send K*N amount of data. This way master only sends K, and then as Greg
>> said all workers work in parallel - each sending additional K (or more
>> precisely K/N * (N-1)).
>>
>> Does this make it clearer?
>>
>> Maja
>>
>> On 5/29/13 9:22 AM, "Greg Malewicz" <malewicz@fb.com> wrote:
>>
>> >Consider parallel time.
>> >
>> >Greg
>> >
>> >On 5/29/13 9:08 AM, "Claudio Martella" <claudio.martella@gmail.com>
>> wrote:
>> >
>> >>Hi,
>> >>
>> >>I have a question about the design of shared aggregators.
>> >>Documentation says:
>> >>
>> >>"After MasterCompute.compute, master doesn't do the distribution of
>>all
>> >>aggregators to all workers, but aggregators again have their owners.
>> >>Master
>> >>only sends each aggregator to its owner, and then each worker
>>distributes
>> >>the aggregators which it owns to all other workers."
>> >>
>> >>Why are the aggregator values not sent directly by the master to all
>> >>workers, instead of doing the two hops it does now?
>> >>
>> >>Suppose I have K aggregators and N workers, the current design
>>requires K
>> >>+
>> >>K*N messages.
>> >>If the master would send the aggregators values to all the workers
>> >>directly, we would have K*N messages, or actually N messages with K
>> >>values
>> >>each.
>> >>
>> >>Am I missing something?
>> >>
>> >>Best,
>> >>Claudio
>> >>
>> >>--
>> >>   Claudio Martella
>> >>   claudio.martella@gmail.com
>> >
>>
>>
>
>
>--=20
>   Claudio Martella
>   claudio.martella@gmail.com