incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chia-Hung Lin <cli...@googlemail.com>
Subject Re: Add counting feature to get the total number of message exchanges
Date Fri, 25 Nov 2011 02:16:55 GMT
+1 for pluggable feature/  hadoop's counter arch

On 25 November 2011 01:05, Thomas Jungblut
<thomas.jungblut@googlemail.com> wrote:
> Great, I open an issue for this counter stuff.
>
> 2011/11/24 Edward J. Yoon <edwardyoon@apache.org>
>
>> >> I think it would be good as a 'pluggable' feature as this counting
>> could>> impact performances.
>> and
>>
>> > It is difficult to say whether we actually need this.*
>> > *In case of SSSP it is useful, but as Tommaso says it can degrade
>>
>> I agree with you. If possible, let's provide as a optional feature.
>>
>> > What do you think of a similar architecture of Counters like in Hadoop?
>>
>> Good.
>>
>> Currently, our Job client reports only shows the total number of
>> supersteps. I think that the Job reports needs to be strengthened. :-)
>>
>> On Thu, Nov 24, 2011 at 11:05 PM, Thomas Jungblut
>> <thomas.jungblut@googlemail.com> wrote:
>> > It is difficult to say whether we actually need this.*
>> > *In case of SSSP it is useful, but as Tommaso says it can degrade
>> > performance for every other BSP by increasing the cost of the sync.
>> >
>> > What do you think of a similar architecture of Counters like in Hadoop?
>> > This could contain even more information, e.G. the number of bytes
>> > transferred etc.
>> >
>> > 2011/11/24 Tommaso Teofili <tommaso.teofili@gmail.com>
>> >
>> >> Hello Edward,
>> >>
>> >> I think it would be good as a 'pluggable' feature as this counting could
>> >> impact performances.
>> >> So it'd be useful when explicitly "debugging" while it should be, in my
>> >> opinion, disabled by default.
>> >>
>> >> My 2 cents.
>> >> Tommaso
>> >>
>> >>
>> >> 2011/11/24 Edward J. Yoon <edwardyoon@apache.org>
>> >>
>> >> > Hi all,
>> >> >
>> >> > I'd like to add counting feature to get the total number of message
>> >> > exchanges during one synchronization among BSP tasks. Before leave
>> >> > barrier in sync() method, each task can report their counts.
>> >> >
>> >> > This information will be useful to user and, some applications (for
>> >> > example, the sync cost of current SSSP example can be reduced to 1/3).
>> >> > Should we add this feature?
>> >> >
>> >> > --
>> >> > Best Regards, Edward J. Yoon
>> >> > @eddieyoon
>> >> >
>> >>
>> >
>> >
>> >
>> > --
>> > Thomas Jungblut
>> > Berlin <thomas.jungblut@gmail.com>
>> >
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
>
>
>
> --
> Thomas Jungblut
> Berlin <thomas.jungblut@gmail.com>
>

Mime
View raw message