hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anastasis Andronidis <andronat_...@hotmail.com>
Subject Re: Bug in Aggregator
Date Thu, 23 Jan 2014 10:38:01 GMT
So you do agree that the user should handle such cases or Hama should implement different methods
to keep track of previous states?

On 23 Ιαν 2014, at 11:31 π.μ., Edward J. Yoon <edwardyoon@apache.org> wrote:

> Yes, AbsDiffAggregator. I've uploaded a patch.
> 
> On Thu, Jan 23, 2014 at 7:21 PM, Anastasis Andronidis
> <andronat_asf@hotmail.com> wrote:
>> Hello,
>> 
>> sorry I am a little bit confused.
>> if I understand correctly, the problem is that we don't keep the value of the previous
superstep? Isn't this something that the user should implement on his own?
>> 
>> I'll take as an example the AbsDiffAggregator:
>> 
>> This aggregator is taking as an input, the (old value of the vertex) - (the current
value) and then it sums everything across all vertices. Isn't this the same as making the
diff inside the vertex and then sending the output to a sum aggregator?
>> 
>> (by the user in Vertex class)
>> 
>> --- in the beginning of the code ---
>> startV = this.getValue();
>> 
>> /* Code run */
>> 
>> this.aggregate("sum_aggr", startV - this.getValue());
>> 
>> If I am correct, I think that we should document this kind of behavior, so the users
can use it. I said that I will handle the documentation on the wiki for the aggregators, I
have it on schedule for this weekend.
>> 
>> Cheers,
>> Anastasis
>> 
>> On 23 Ιαν 2014, at 2:30 π.μ., Edward J. Yoon <edwardyoon@apache.org> wrote:
>> 
>>> I found a bug in aggregator.
>>> 
>>> In parseMessages, you calls masterAggregation() method. Do you think
>>> everything is OK?
>>> 
>>> /**
>>>  * Method to let the custom master aggregator read messages from peers and
>>>  * aggregate a value.
>>>  */
>>> @SuppressWarnings("unchecked")
>>> public void masterAggregation(Text name, Writable value) {
>>>   String nameIdx = name.toString().split(";", 2)[1];
>>>   this.Aggregators.get(nameIdx).aggregate(null, value);
>>> 
>>>   // When it's time to send the values, we can see which aggregators are used.
>>>   this.aggregatorsUsed.add(nameIdx);
>>> }
>>> 
>>> The aggregated value will be always the last value.
>>> 
>>> Like our old code,
>>> 
>>> getAggregationRunner().aggregateVertex(lastValue, vertex);
>>> 
>>> You should aggregates all values.
>>> 
>>> --
>>> Best Regards, Edward J. Yoon
>>> @eddieyoon
>>> 
>> 
> 
> 
> 
> -- 
> Best Regards, Edward J. Yoon
> @eddieyoon
> 


Mime
View raw message