airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raminder Singh <raminderjsi...@gmail.com>
Subject Re: Implementing the cancel/terminate
Date Mon, 21 Apr 2014 18:02:30 GMT
Yes, its important to track those job states and make actions based on user request. Job canceling
at the early stage may have few time but it need to be handled in Airavata. Can we use registry
to communicate such status to input chain or provider?   

Thanks
Raminder

On Apr 21, 2014, at 10:51 AM, Saminda Wijeratne <samindaw@gmail.com> wrote:

> hmm... The reason why I thought the GFacProvider should handle is that the actual process
of job submission involves many steps including authentications, data transfers apart from
the actual job submission. Cancellation should be possible inbetween each of those steps.
Thus the active provider should monitor cancellation requests and stop whatever its doing.
If the job is already submitted I think the Monitoring which you mentioned could take over
the cancellation. wdyt?
> 
> 
> On Mon, Apr 21, 2014 at 7:35 AM, Raminder Singh <raminderjsingh@gmail.com> wrote:
>  Thanks for investigation the problem and working through solution. This is really required
for the production gateways like Ultrascan.
> 
> In the current architecture where we have job submission(provider) and monitoring separate,
job cancel request need not to go to GFAC provider. Provider submits the jobs and handover
the job id to the orchestrator. Orchestrator works with the job monitoring to maintain the
job state. Now the cancel need to be handled by Orchestrator and Monitoring. That will change
the course of action for API to cancel a job. 
> 
> One important requirement to take care is cleanup task after the job is canceled like
updating the job status table and updating the status. 
> 
> Thanks
> Raminder 
> 
> 
> On Apr 21, 2014, at 9:06 AM, Saminda Wijeratne <samindaw@gmail.com> wrote:
> 
>>  Hi All,
>> 
>> After looking at the current design and doing some trial and error I thought of implementing
the cancellation as follows.
>> 
>> Cancellation of an experiment requested by a gateway requires cancellation request
to go through several layers. (Orchestrator > GFac > GFac Provider)
>> Each layer is responsible for handling cancellation relevant for that layer (Orchestration
cancels experiment, GFac cancels Task, GFac Provider cancels Job)
>> What I thought is, each layer will listen to cancellation request made to the layer
above and perform its cancellation actions accordingly. (GFac will see the experiment is having
the status "canceling" for an experiment id and it will perform cancellation of the tasks
relevant for that experiment)
>> Effectively the Orchestrator will be 
>> updating the status of the experiment in registry with the status "canceling" 
>> publish a message which will be caught by GFac instance which handles its Tasks.

>> GFac will perform the same and the correct GFac Provider instance will catch the
message and perform the actual job cancellation.
>> Once the job cancellation is done the statuses at each layer will be updated (to
"canceled") in  similar fashion.
>> We allow the API call of cancellation to be asynchronous
>> I'm hoping to use the MonitorPublisher implemented by Lahiru to publish the messages.
>> wdyt?
>> 
>> 
>> 
>> Thanks,
>> 
>> Saminda
>> 
>> 
> 
> 


Mime
View raw message