hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tommaso Teofili <tommaso.teof...@gmail.com>
Subject Re: Please review new APIs.
Date Wed, 02 Nov 2011 12:09:07 GMT
2011/11/2 Edward J. Yoon <edwardyoon@apache.org>

> > I'm sure that not every job actually needs a cleanup or a setup.
>
> You're right. Almost BSP applications should override bsp() method
> but, setup() and cleaner() methods are not as you said. Let's fix
> them.
>

Agreed +1


>
> > Generally I would suggest to integrate the OutputCollector and the
> > RecordReader into the BSPPeerImpl.
> > So our peer is like the context in Hadoop.
>
> Good idea.
>

+1 here too

Tommaso


>
> On Wed, Nov 2, 2011 at 9:03 PM, Thomas Jungblut
> <thomas.jungblut@googlemail.com> wrote:
> > Yes. When I reworked that API, I made a default implementation in our
> > abstract BSP class.
> > So the user has to override the methods for himself, if he needs to.
> > I'm sure that not every job actually needs a cleanup or a setup.
> >
> > Generally I would suggest to integrate the OutputCollector and the
> > RecordReader into the BSPPeerImpl.
> > So our peer is like the context in Hadoop.
> > But that is just a minor thing. It is a great improvement ;)
> >
> > 2011/11/2 Edward J. Yoon <edwardyoon@apache.org>
> >
> >> There're bsp(), setup() and cleaner() methods.
> >>
> >> What is you suggestion?
> >>
> >> On Wed, Nov 2, 2011 at 8:47 PM, Thomas Jungblut
> >> <thomas.jungblut@googlemail.com> wrote:
> >> > Have a look at the combiner class. I know that this is just a "test",
> but
> >> > it is really messy if the user does not use the methods, but is
> forced to
> >> > override them.
> >> >
> >> > 2011/11/2 Edward J. Yoon <edwardyoon@apache.org>
> >> >
> >> >> Why?
> >> >>
> >> >> On Wed, Nov 2, 2011 at 8:21 PM, Thomas Jungblut
> >> >> <thomas.jungblut@googlemail.com> wrote:
> >> >> > I totally dislike that BSP class now has abstract methods instead
> of
> >> >> > default implementations.
> >> >> >
> >> >> > 2011/11/2 Edward J. Yoon <edwardyoon@apache.org>
> >> >> >
> >> >> >> Hi all,
> >> >> >>
> >> >> >> As you know, recently combiners and IO are added.
> >> >> >>
> >> >> >> Please review them from user viewpoint.
> >> >> >>
> >> >> >>
> >> >> >>
> >> >>
> >>
> http://svn.apache.org/repos/asf/incubator/hama/trunk/examples/src/main/java/org/apache/hama/examples/PiEstimator.java
> >> >> >>
> >> >> >> I'm testing multiple tasks and IO features on 100 nodes cluster
> using
> >> >> >> 10 tasks per node. If there's no issue, I'll close HAMA-258.
> >> >> >>
> >> >> >> Thanks.
> >> >> >>
> >> >> >> --
> >> >> >> Best Regards, Edward J. Yoon
> >> >> >> @eddieyoon
> >> >> >>
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Thomas Jungblut
> >> >> > Berlin <thomas.jungblut@gmail.com>
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Best Regards, Edward J. Yoon
> >> >> @eddieyoon
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Thomas Jungblut
> >> > Berlin <thomas.jungblut@gmail.com>
> >> >
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >>
> >
> >
> >
> > --
> > Thomas Jungblut
> > Berlin <thomas.jungblut@gmail.com>
> >
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message