Mailing-List: contact dev-help@hama.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hama.apache.org
Received-SPF: pass (athena.apache.org: domain of andronat_asf@hotmail.com
 designates 65.55.111.141 as permitted sender)
Message-ID: <BLU0-SMTP3156A0C41137674ED15022E9A60@phx.gbl>
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0 (Mac OS X Mail 7.1 \(1827\))
Subject: Re: Bug in Aggregator
From: Anastasis Andronidis <andronat_asf@hotmail.com>
In-Reply-To: 
 <CAGQgZQSwzqkPtb-hnPgwdskEQiXZOxHdfk3+YUCcyAAL45Pieg@mail.gmail.com>
Date: Thu, 23 Jan 2014 11:38:01 +0100
Content-Transfer-Encoding: quoted-printable
References: 
 <CAGQgZQRz9tPOON+aepEX2sMDTg=i4SiUOgTn6aQKb_455M7dBA@mail.gmail.com>
 <BLU0-SMTP23489649C12C05755FB2A46E9A60@phx.gbl>
 <CAGQgZQSwzqkPtb-hnPgwdskEQiXZOxHdfk3+YUCcyAAL45Pieg@mail.gmail.com>
To: dev@hama.apache.org

So you do agree that the user should handle such cases or Hama should =
implement different methods to keep track of previous states?

On 23 =CE=99=CE=B1=CE=BD 2014, at 11:31 =CF=80.=CE=BC., Edward J. Yoon =
<edwardyoon@apache.org> wrote:

> Yes, AbsDiffAggregator. I've uploaded a patch.
>=20
> On Thu, Jan 23, 2014 at 7:21 PM, Anastasis Andronidis
> <andronat_asf@hotmail.com> wrote:
>> Hello,
>>=20
>> sorry I am a little bit confused.
>> if I understand correctly, the problem is that we don't keep the =
value of the previous superstep? Isn't this something that the user =
should implement on his own?
>>=20
>> I'll take as an example the AbsDiffAggregator:
>>=20
>> This aggregator is taking as an input, the (old value of the vertex) =
- (the current value) and then it sums everything across all vertices. =
Isn't this the same as making the diff inside the vertex and then =
sending the output to a sum aggregator?
>>=20
>> (by the user in Vertex class)
>>=20
>> --- in the beginning of the code ---
>> startV =3D this.getValue();
>>=20
>> /* Code run */
>>=20
>> this.aggregate("sum_aggr", startV - this.getValue());
>>=20
>> If I am correct, I think that we should document this kind of =
behavior, so the users can use it. I said that I will handle the =
documentation on the wiki for the aggregators, I have it on schedule for =
this weekend.
>>=20
>> Cheers,
>> Anastasis
>>=20
>> On 23 =CE=99=CE=B1=CE=BD 2014, at 2:30 =CF=80.=CE=BC., Edward J. Yoon =
<edwardyoon@apache.org> wrote:
>>=20
>>> I found a bug in aggregator.
>>>=20
>>> In parseMessages, you calls masterAggregation() method. Do you think
>>> everything is OK?
>>>=20
>>> /**
>>>  * Method to let the custom master aggregator read messages from =
peers and
>>>  * aggregate a value.
>>>  */
>>> @SuppressWarnings("unchecked")
>>> public void masterAggregation(Text name, Writable value) {
>>>   String nameIdx =3D name.toString().split(";", 2)[1];
>>>   this.Aggregators.get(nameIdx).aggregate(null, value);
>>>=20
>>>   // When it's time to send the values, we can see which aggregators =
are used.
>>>   this.aggregatorsUsed.add(nameIdx);
>>> }
>>>=20
>>> The aggregated value will be always the last value.
>>>=20
>>> Like our old code,
>>>=20
>>> getAggregationRunner().aggregateVertex(lastValue, vertex);
>>>=20
>>> You should aggregates all values.
>>>=20
>>> --
>>> Best Regards, Edward J. Yoon
>>> @eddieyoon
>>>=20
>>=20
>=20
>=20
>=20
> --=20
> Best Regards, Edward J. Yoon
> @eddieyoon
>=20