hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Please review new APIs.
Date Wed, 02 Nov 2011 12:07:50 GMT
> I'm sure that not every job actually needs a cleanup or a setup.

You're right. Almost BSP applications should override bsp() method
but, setup() and cleaner() methods are not as you said. Let's fix
them.

> Generally I would suggest to integrate the OutputCollector and the
> RecordReader into the BSPPeerImpl.
> So our peer is like the context in Hadoop.

Good idea.

On Wed, Nov 2, 2011 at 9:03 PM, Thomas Jungblut
<thomas.jungblut@googlemail.com> wrote:
> Yes. When I reworked that API, I made a default implementation in our
> abstract BSP class.
> So the user has to override the methods for himself, if he needs to.
> I'm sure that not every job actually needs a cleanup or a setup.
>
> Generally I would suggest to integrate the OutputCollector and the
> RecordReader into the BSPPeerImpl.
> So our peer is like the context in Hadoop.
> But that is just a minor thing. It is a great improvement ;)
>
> 2011/11/2 Edward J. Yoon <edwardyoon@apache.org>
>
>> There're bsp(), setup() and cleaner() methods.
>>
>> What is you suggestion?
>>
>> On Wed, Nov 2, 2011 at 8:47 PM, Thomas Jungblut
>> <thomas.jungblut@googlemail.com> wrote:
>> > Have a look at the combiner class. I know that this is just a "test", but
>> > it is really messy if the user does not use the methods, but is forced to
>> > override them.
>> >
>> > 2011/11/2 Edward J. Yoon <edwardyoon@apache.org>
>> >
>> >> Why?
>> >>
>> >> On Wed, Nov 2, 2011 at 8:21 PM, Thomas Jungblut
>> >> <thomas.jungblut@googlemail.com> wrote:
>> >> > I totally dislike that BSP class now has abstract methods instead of
>> >> > default implementations.
>> >> >
>> >> > 2011/11/2 Edward J. Yoon <edwardyoon@apache.org>
>> >> >
>> >> >> Hi all,
>> >> >>
>> >> >> As you know, recently combiners and IO are added.
>> >> >>
>> >> >> Please review them from user viewpoint.
>> >> >>
>> >> >>
>> >> >>
>> >>
>> http://svn.apache.org/repos/asf/incubator/hama/trunk/examples/src/main/java/org/apache/hama/examples/PiEstimator.java
>> >> >>
>> >> >> I'm testing multiple tasks and IO features on 100 nodes cluster
using
>> >> >> 10 tasks per node. If there's no issue, I'll close HAMA-258.
>> >> >>
>> >> >> Thanks.
>> >> >>
>> >> >> --
>> >> >> Best Regards, Edward J. Yoon
>> >> >> @eddieyoon
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Thomas Jungblut
>> >> > Berlin <thomas.jungblut@gmail.com>
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Best Regards, Edward J. Yoon
>> >> @eddieyoon
>> >>
>> >
>> >
>> >
>> > --
>> > Thomas Jungblut
>> > Berlin <thomas.jungblut@gmail.com>
>> >
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
>
>
>
> --
> Thomas Jungblut
> Berlin <thomas.jungblut@gmail.com>
>



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Mime
View raw message