airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Danushka Menikkumbura <danushka.menikkumb...@gmail.com>
Subject Re: Implementing the cancel/terminate
Date Tue, 29 Apr 2014 20:49:30 GMT
To me this sounds a little more complicated.If we take the Whirr-based
cloud bursting impl for instance,we have an in handler to launch a cluster
and an out handler to shut it down.When a job is cancelled in a case like
this, the corresponding cluster should be shut down and for that the out
handler needs to be notified.

/Danushka

On Tuesday, April 29, 2014, Lahiru Gunathilake <glahiru@gmail.com> wrote:
> Hi Marlon,
>
> On Mon, Apr 28, 2014 at 3:59 PM, Marlon Pierce <marpierc@iu.edu> wrote:
>>
>> Are any handlers called when canceling? Should they be?  Seems like you
>> would need to typically do some cleanup.
>
> +1 thats the plan to have pre/post handlers for terminate operation like
we have to submit.
> Lahiru
>>
>> Marlon
>>
>> On 4/23/14 2:35 PM, Saminda Wijeratne wrote:
>> > Got it Suresh. Thanks for putting it clearly :)
>> >
>> >
>> > On Tue, Apr 22, 2014 at 5:48 PM, Suresh Marru <smarru@apache.org>
wrote:
>> >
>> >> On Tue, Apr 22, 2014 at 6:26 PM, Saminda Wijeratne <samindaw@gmail.com
>wrote:
>> >>
>> >>> You mean like a monitoring client at client end triggering a
listener?
>> >>>
>> >> Thats ideal, but different story altogether. I mean triggering a
listener
>> >> for all major status changes including canceling.
>> >>
>> >> In this context I mean once the client gets Canceling at experiment
level,
>> >> once all lower level statues gets updated, we should ensure subsequent
>> >> client query to experiment returned Cancelled. I see your plan already
>> >> accounts for it, was just restating its simpler for gateways to wait
on
>> >> experiment status instead of expecting to navigate through Task->Job
to
>> >> verify the success.
>> >>
>> >> Suresh
>> >>
>> >>
>> >>>
>> >>> On Tue, Apr 22, 2014 at 5:22 PM, Suresh Marru <smarru@apache.org>
wrote:
>> >>>
>> >>>> Hi Saminda,
>> >>>>
>> >>>> This like a good plan, I think we should make sure the API server
>> >>>> properly reports these statuses back to the clients.
>> >>>>
>> >>>> Suresh
>> >>>>
>> >>>> On Apr 21, 2014, at 9:06 AM, Saminda Wijeratne <samindaw@gmail.com>
>> >>>> wrote:
>> >>>>
>> >>>>  Hi All,
>> >>>>
>> >>>> After looking at the current design and doing some trial and error
I
>> >>>> thought of implementing the cancellation as follows.
>> >>>>
>> >>>>
>> >>>>    - Cancellation of an experiment requested by a gateway requires
>> >>>>    cancellation request to go through several layers. (Orchestrator
> GFac >
>> >>>>    GFac Provider)
>> >>>>    - Each layer is responsible for handling cancellation relevant
for
>> >>>>    that layer (Orchestration cancels experiment, GFac cancels Task,
GFac
>> >>>>    Provider cancels Job)
>> >>>>    - What I thought is, each layer will listen to cancellation
request
>> >>>>    made to the layer above and perform its cancellation actions
accordingly.
>> >>>>    (GFac will see the experiment is having the status "canceling"
for an
>> >>>>    experiment id and it will perform cancellation of the tasks
relevant for
>> >>>>    that experiment)
>> >>>>       - Effectively the Orchestrator will be
>> >>>>       - updating the status of the experiment in registry with the
>> >>>>          status "canceling"
>> >>>>          - publish a message which will be caught by GFac instance
>> >>>>          which handles its Tasks.
>> >>>>          - GFac will perform the same and the correct GFac Provider
>> >>>>       instance will catch the message and perform the actual job
cancellation.
>> >>>>    - Once the job cancellation is done the statuses at each layer
will
>> >>>>    be updated (to "canceled") in  similar fashion.
>> >>>>    - We allow the API call of cancellation to be asynchronous
>> >>>>    - I'm hoping to use the MonitorPublisher implemented by Lahiru
to
>> >>>>    publish the messages.
>> >>>>
>> >>>> wdyt?
>> >>>>
>> >>>>
>> >>>> Thanks,
>> >>>>
>> >>>> Saminda
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>>
>
>
>
> --
> System Analyst Programmer
> PTI Lab
> Indiana University

Mime
View raw message