hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Jungblut <thomas.jungb...@googlemail.com>
Subject Re: [DISCUSS] Chaining Supersteps setup, bsp and cleanup execution order
Date Mon, 21 May 2012 16:58:38 GMT
So let's discuss about it.
I have to agree that the documentation in the superstep class is misleading:

Setup this superstep, is called once before compute().


The question is, if that really matters. I think we shouldn't hand this to
the user, this is a too unstable API.
I'm sure it will change if we roll out the whole FT stuff.

2012/5/21 Suraj Menon <menonsuraj5@gmail.com>

> Yeah, I looked into the code, that's how figured it. Not sure how many
> users look into the source code and I agree with you; I ended up doing
> the same. I started this discussion because, I wanted to get a general
> opinion.
> I don't know how much of our opinion matters because we have been
> involved in the design and implementation. Just wanted to get reviews
> from other devs and users. Knowing the code, I am happy either ways
> :). We should make it a point to document this whichever way we
> choose. If we change this in future it will not cause any compilation
> error but might break the logic of existing programs.
>
> -Suraj
>
> On Mon, May 21, 2012 at 11:54 AM, Thomas Jungblut
> <thomas.jungblut@googlemail.com> wrote:
> > Well then you should have looked into the SuperstepBSP class before you
> > started coding :)
> > No seriously, I think it isn't very counter intuitive, if you want to
> call
> > things before computation you can simply put it into the beginning of
> > computation.
> > Setup is for setting up some configuration values or resources, thus
> > initialized at the start for all supersteps.
> >
> > 2012/5/21 Suraj Menon <menonsuraj5@gmail.com>
> >
> >> As I said, my setup of second Superstep assumed that the Superstep1's
> >> computation is completed.
> >> Cleanup should be fine, because wherever we do it, we are sure that
> >> the resource is no longer used in the system.
> >>
> >> On Mon, May 21, 2012 at 10:30 AM, Thomas Jungblut
> >> <thomas.jungblut@googlemail.com> wrote:
> >> > I thought it is not more than just a job cleanup. So actually just the
> >> last
> >> > superstep should be interested in cleaning up. But there may be
> resources
> >> > in other steps that should be cleaned up as well.
> >> > However, Why do you need Order2 if you can simply put the code in
> cleanup
> >> > to the end of the compute function? Same with setup.
> >> >
> >> > 2012/5/21 Suraj Menon <surajsmenon@apache.org>
> >> >
> >> >> Hi, I wanted to open this discussion on the order by which the setup,
> >> >> bsp and cleanup functions are called in Chaining Supestep design.
> >> >>
> >> >> Today the execution order is (say for 3 supersteps defined for a
> task) :
> >> >>
> >> >> **Order1**
> >> >>
> >> >> 1. Superstep1.setup
> >> >> 2. Superstep2.setup
> >> >> 3. Superstep3.setup
> >> >> 4. Superstep1.compute
> >> >> 5. Superstep2.compute
> >> >> 6. Superstep3.compute
> >> >> 7. Superstep1.cleanup
> >> >> 8. Superstep2.cleanup
> >> >> 9. Superstep3.cleanup
> >> >>
> >> >> However, I had a totally different intuition on this. I assumed that
> >> >> every Superstep lifecycle is completely executed before the next one
> >> >> in chain.
> >> >> I programmed setup of Superstep2 to be dependent on results in
> >> >> Superstep1#compute function, which turned out to be buggy.
> >> >>
> >> >>
> >> >> **Order2**
> >> >>
> >> >> 1. Superstep1.setup
> >> >> 2. Superstep1.compute
> >> >> 3. Superstep1.cleanup
> >> >> 4. Superstep2.setup
> >> >> 5. Superstep2.compute
> >> >> 6. Superstep2.cleanup
> >> >> 7. Superstep3.setup
> >> >> 8. Superstep3.bsp
> >> >> 9. Superstep3.cleanup
> >> >>
> >> >> This might be counter-intuitive for some others. Looking for opinions
> >> >> on which execution order to choose.
> >> >> We have not released it yet, hence we can make a decision in time.
> >> >>
> >> >>
> >> >> Thanks,
> >> >> Suraj
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Thomas Jungblut
> >> > Berlin <thomas.jungblut@gmail.com>
> >>
> >
> >
> >
> > --
> > Thomas Jungblut
> > Berlin <thomas.jungblut@gmail.com>
>



-- 
Thomas Jungblut
Berlin <thomas.jungblut@gmail.com>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message