incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suraj Menon <menonsur...@gmail.com>
Subject Re: [DISCUSS] Chaining Supersteps setup, bsp and cleanup execution order
Date Mon, 21 May 2012 16:29:35 GMT
Yeah, I looked into the code, that's how figured it. Not sure how many
users look into the source code and I agree with you; I ended up doing
the same. I started this discussion because, I wanted to get a general
opinion.
I don't know how much of our opinion matters because we have been
involved in the design and implementation. Just wanted to get reviews
from other devs and users. Knowing the code, I am happy either ways
:). We should make it a point to document this whichever way we
choose. If we change this in future it will not cause any compilation
error but might break the logic of existing programs.

-Suraj

On Mon, May 21, 2012 at 11:54 AM, Thomas Jungblut
<thomas.jungblut@googlemail.com> wrote:
> Well then you should have looked into the SuperstepBSP class before you
> started coding :)
> No seriously, I think it isn't very counter intuitive, if you want to call
> things before computation you can simply put it into the beginning of
> computation.
> Setup is for setting up some configuration values or resources, thus
> initialized at the start for all supersteps.
>
> 2012/5/21 Suraj Menon <menonsuraj5@gmail.com>
>
>> As I said, my setup of second Superstep assumed that the Superstep1's
>> computation is completed.
>> Cleanup should be fine, because wherever we do it, we are sure that
>> the resource is no longer used in the system.
>>
>> On Mon, May 21, 2012 at 10:30 AM, Thomas Jungblut
>> <thomas.jungblut@googlemail.com> wrote:
>> > I thought it is not more than just a job cleanup. So actually just the
>> last
>> > superstep should be interested in cleaning up. But there may be resources
>> > in other steps that should be cleaned up as well.
>> > However, Why do you need Order2 if you can simply put the code in cleanup
>> > to the end of the compute function? Same with setup.
>> >
>> > 2012/5/21 Suraj Menon <surajsmenon@apache.org>
>> >
>> >> Hi, I wanted to open this discussion on the order by which the setup,
>> >> bsp and cleanup functions are called in Chaining Supestep design.
>> >>
>> >> Today the execution order is (say for 3 supersteps defined for a task) :
>> >>
>> >> **Order1**
>> >>
>> >> 1. Superstep1.setup
>> >> 2. Superstep2.setup
>> >> 3. Superstep3.setup
>> >> 4. Superstep1.compute
>> >> 5. Superstep2.compute
>> >> 6. Superstep3.compute
>> >> 7. Superstep1.cleanup
>> >> 8. Superstep2.cleanup
>> >> 9. Superstep3.cleanup
>> >>
>> >> However, I had a totally different intuition on this. I assumed that
>> >> every Superstep lifecycle is completely executed before the next one
>> >> in chain.
>> >> I programmed setup of Superstep2 to be dependent on results in
>> >> Superstep1#compute function, which turned out to be buggy.
>> >>
>> >>
>> >> **Order2**
>> >>
>> >> 1. Superstep1.setup
>> >> 2. Superstep1.compute
>> >> 3. Superstep1.cleanup
>> >> 4. Superstep2.setup
>> >> 5. Superstep2.compute
>> >> 6. Superstep2.cleanup
>> >> 7. Superstep3.setup
>> >> 8. Superstep3.bsp
>> >> 9. Superstep3.cleanup
>> >>
>> >> This might be counter-intuitive for some others. Looking for opinions
>> >> on which execution order to choose.
>> >> We have not released it yet, hence we can make a decision in time.
>> >>
>> >>
>> >> Thanks,
>> >> Suraj
>> >>
>> >
>> >
>> >
>> > --
>> > Thomas Jungblut
>> > Berlin <thomas.jungblut@gmail.com>
>>
>
>
>
> --
> Thomas Jungblut
> Berlin <thomas.jungblut@gmail.com>

Mime
View raw message