airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marlon Pierce <marpi...@iu.edu>
Subject Re: Implementing the cancel/terminate
Date Tue, 29 Apr 2014 20:58:58 GMT
I assumed there would be a lot of overlap between cancel-handlers and
out-handlers but not complete overlap.  Cleanup operations need to be
done in either case, but "move the successfully created output files to
a safe storage location" would not be done.

Marlon

On 4/29/14 4:49 PM, Danushka Menikkumbura wrote:
> To me this sounds a little more complicated.If we take the Whirr-based
> cloud bursting impl for instance,we have an in handler to launch a cluster
> and an out handler to shut it down.When a job is cancelled in a case like
> this, the corresponding cluster should be shut down and for that the out
> handler needs to be notified.
>
> /Danushka
>
> On Tuesday, April 29, 2014, Lahiru Gunathilake <glahiru@gmail.com> wrote:
>> Hi Marlon,
>>
>> On Mon, Apr 28, 2014 at 3:59 PM, Marlon Pierce <marpierc@iu.edu> wrote:
>>> Are any handlers called when canceling? Should they be?  Seems like you
>>> would need to typically do some cleanup.
>> +1 thats the plan to have pre/post handlers for terminate operation like
> we have to submit.
>> Lahiru
>>> Marlon
>>>
>>> On 4/23/14 2:35 PM, Saminda Wijeratne wrote:
>>>> Got it Suresh. Thanks for putting it clearly :)
>>>>
>>>>
>>>> On Tue, Apr 22, 2014 at 5:48 PM, Suresh Marru <smarru@apache.org>
> wrote:
>>>>> On Tue, Apr 22, 2014 at 6:26 PM, Saminda Wijeratne <samindaw@gmail.com
>> wrote:
>>>>>> You mean like a monitoring client at client end triggering a
> listener?
>>>>> Thats ideal, but different story altogether. I mean triggering a
> listener
>>>>> for all major status changes including canceling.
>>>>>
>>>>> In this context I mean once the client gets Canceling at experiment
> level,
>>>>> once all lower level statues gets updated, we should ensure subsequent
>>>>> client query to experiment returned Cancelled. I see your plan already
>>>>> accounts for it, was just restating its simpler for gateways to wait
> on
>>>>> experiment status instead of expecting to navigate through Task->Job
> to
>>>>> verify the success.
>>>>>
>>>>> Suresh
>>>>>
>>>>>
>>>>>> On Tue, Apr 22, 2014 at 5:22 PM, Suresh Marru <smarru@apache.org>
> wrote:
>>>>>>> Hi Saminda,
>>>>>>>
>>>>>>> This like a good plan, I think we should make sure the API server
>>>>>>> properly reports these statuses back to the clients.
>>>>>>>
>>>>>>> Suresh
>>>>>>>
>>>>>>> On Apr 21, 2014, at 9:06 AM, Saminda Wijeratne <samindaw@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>  Hi All,
>>>>>>>
>>>>>>> After looking at the current design and doing some trial and
error I
>>>>>>> thought of implementing the cancellation as follows.
>>>>>>>
>>>>>>>
>>>>>>>    - Cancellation of an experiment requested by a gateway requires
>>>>>>>    cancellation request to go through several layers. (Orchestrator
>> GFac >
>>>>>>>    GFac Provider)
>>>>>>>    - Each layer is responsible for handling cancellation relevant
> for
>>>>>>>    that layer (Orchestration cancels experiment, GFac cancels
Task,
> GFac
>>>>>>>    Provider cancels Job)
>>>>>>>    - What I thought is, each layer will listen to cancellation
> request
>>>>>>>    made to the layer above and perform its cancellation actions
> accordingly.
>>>>>>>    (GFac will see the experiment is having the status "canceling"
> for an
>>>>>>>    experiment id and it will perform cancellation of the tasks
> relevant for
>>>>>>>    that experiment)
>>>>>>>       - Effectively the Orchestrator will be
>>>>>>>       - updating the status of the experiment in registry with
the
>>>>>>>          status "canceling"
>>>>>>>          - publish a message which will be caught by GFac instance
>>>>>>>          which handles its Tasks.
>>>>>>>          - GFac will perform the same and the correct GFac Provider
>>>>>>>       instance will catch the message and perform the actual
job
> cancellation.
>>>>>>>    - Once the job cancellation is done the statuses at each layer
> will
>>>>>>>    be updated (to "canceled") in  similar fashion.
>>>>>>>    - We allow the API call of cancellation to be asynchronous
>>>>>>>    - I'm hoping to use the MonitorPublisher implemented by Lahiru
to
>>>>>>>    publish the messages.
>>>>>>>
>>>>>>> wdyt?
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Saminda
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>
>>
>> --
>> System Analyst Programmer
>> PTI Lab
>> Indiana University


Mime
View raw message