Mailing-List: contact dev-help@hama.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hama.apache.org
Received-SPF: pass (athena.apache.org: domain of andronat_asf@hotmail.com
 designates 65.55.111.147 as permitted sender)
Message-ID: <BLU0-SMTP4728C3010ADAD870BF5B17CE9A60@phx.gbl>
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0 (Mac OS X Mail 7.1 \(1827\))
Subject: Re: Bug in Aggregator
From: Anastasis Andronidis <andronat_asf@hotmail.com>
In-Reply-To: 
 <CAGQgZQR_+477UxSgqV4_Mt=O2X-=Hi2QWW7n1m0DeqTEip1R8g@mail.gmail.com>
Date: Thu, 23 Jan 2014 14:53:08 +0100
Content-Transfer-Encoding: quoted-printable
References: 
 <CAGQgZQRz9tPOON+aepEX2sMDTg=i4SiUOgTn6aQKb_455M7dBA@mail.gmail.com>
 <BLU0-SMTP23489649C12C05755FB2A46E9A60@phx.gbl>
 <CAGQgZQSwzqkPtb-hnPgwdskEQiXZOxHdfk3+YUCcyAAL45Pieg@mail.gmail.com>
 <BLU0-SMTP3156A0C41137674ED15022E9A60@phx.gbl>
 <CAGQgZQR_+477UxSgqV4_Mt=O2X-=Hi2QWW7n1m0DeqTEip1R8g@mail.gmail.com>
To: dev@hama.apache.org

Sorry, can't see something easy and elegant. For me, the best approach =
would be to stay on simple aggregate(name, value).

BTW, my patch has other parts that simplify the code. Are you sure that =
it's all written wrong? Should I open smaller tickets to do updates? Or =
just drop everything? (e.g. named aggregators, etc..)

The AbsDiffAggregator is a complicate case. I don't see any convenient =
way to support it with a simple interface like vertex.aggregate(name, =
value); I don't see a way to interact it with=20

On 23 =CE=99=CE=B1=CE=BD 2014, at 12:14 =CE=BC.=CE=BC., Edward J. Yoon =
<edwardyoon@apache.org> wrote:

> Yes, basically I agree with you. User should handle it. BTW, at the
> same time, I don't want to remove AbsDiffAggregator.
>=20
> Do you have any good idea?
>=20
> On Thu, Jan 23, 2014 at 7:38 PM, Anastasis Andronidis
> <andronat_asf@hotmail.com> wrote:
>> So you do agree that the user should handle such cases or Hama should =
implement different methods to keep track of previous states?
>>=20
>> On 23 =CE=99=CE=B1=CE=BD 2014, at 11:31 =CF=80.=CE=BC., Edward J. =
Yoon <edwardyoon@apache.org> wrote:
>>=20
>>> Yes, AbsDiffAggregator. I've uploaded a patch.
>>>=20
>>> On Thu, Jan 23, 2014 at 7:21 PM, Anastasis Andronidis
>>> <andronat_asf@hotmail.com> wrote:
>>>> Hello,
>>>>=20
>>>> sorry I am a little bit confused.
>>>> if I understand correctly, the problem is that we don't keep the =
value of the previous superstep? Isn't this something that the user =
should implement on his own?
>>>>=20
>>>> I'll take as an example the AbsDiffAggregator:
>>>>=20
>>>> This aggregator is taking as an input, the (old value of the =
vertex) - (the current value) and then it sums everything across all =
vertices. Isn't this the same as making the diff inside the vertex and =
then sending the output to a sum aggregator?
>>>>=20
>>>> (by the user in Vertex class)
>>>>=20
>>>> --- in the beginning of the code ---
>>>> startV =3D this.getValue();
>>>>=20
>>>> /* Code run */
>>>>=20
>>>> this.aggregate("sum_aggr", startV - this.getValue());
>>>>=20
>>>> If I am correct, I think that we should document this kind of =
behavior, so the users can use it. I said that I will handle the =
documentation on the wiki for the aggregators, I have it on schedule for =
this weekend.
>>>>=20
>>>> Cheers,
>>>> Anastasis
>>>>=20
>>>> On 23 =CE=99=CE=B1=CE=BD 2014, at 2:30 =CF=80.=CE=BC., Edward J. =
Yoon <edwardyoon@apache.org> wrote:
>>>>=20
>>>>> I found a bug in aggregator.
>>>>>=20
>>>>> In parseMessages, you calls masterAggregation() method. Do you =
think
>>>>> everything is OK?
>>>>>=20
>>>>> /**
>>>>> * Method to let the custom master aggregator read messages from =
peers and
>>>>> * aggregate a value.
>>>>> */
>>>>> @SuppressWarnings("unchecked")
>>>>> public void masterAggregation(Text name, Writable value) {
>>>>>  String nameIdx =3D name.toString().split(";", 2)[1];
>>>>>  this.Aggregators.get(nameIdx).aggregate(null, value);
>>>>>=20
>>>>>  // When it's time to send the values, we can see which =
aggregators are used.
>>>>>  this.aggregatorsUsed.add(nameIdx);
>>>>> }
>>>>>=20
>>>>> The aggregated value will be always the last value.
>>>>>=20
>>>>> Like our old code,
>>>>>=20
>>>>> getAggregationRunner().aggregateVertex(lastValue, vertex);
>>>>>=20
>>>>> You should aggregates all values.
>>>>>=20
>>>>> --
>>>>> Best Regards, Edward J. Yoon
>>>>> @eddieyoon
>>>>>=20
>>>>=20
>>>=20
>>>=20
>>>=20
>>> --
>>> Best Regards, Edward J. Yoon
>>> @eddieyoon
>>>=20
>>=20
>=20
>=20
>=20
> --=20
> Best Regards, Edward J. Yoon
> @eddieyoon
>=20