airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amila Jayasekara <thejaka.am...@gmail.com>
Subject Re: Synchronous vs Async submissions in GFAC provider
Date Tue, 16 Jul 2013 18:00:22 GMT
On Tue, Jul 16, 2013 at 1:57 PM, Lahiru Gunathilake <glahiru@gmail.com>wrote:

>
>
>
> On Tue, Jul 16, 2013 at 11:23 PM, Amila Jayasekara <
> thejaka.amila@gmail.com> wrote:
>
>> +1 Raman.
>> Is this going to be in 0.9 ? If so can you please update feature list for
>> 0.9 also ?
>>
>>
>> On Tue, Jul 16, 2013 at 11:58 AM, Raminder Singh <
>> raminderjsingh@gmail.com> wrote:
>>
>>> I am working on a use case where these is an asynchronous job submission
>>> script for pegasus[1] (Condor Dagman [2]). Submission scripts returns a
>>> JOBID to monitor the status.
>>
>>
>> Does pegasus generate above job id ?
>>
>>
>>
>>> Monitor script returns the job status using the JOBID provided by
>>> submission script. Another script is available to cancel the job.
>>>
>>> I plan to use SSHProvider to call these scripts. Currently provider only
>>> support syncronous submission and does not allow to add a wait thread using
>>> another script. According to me, this can be common pattern among other
>>> gateway use cases.  This case is true for LocalProvider also.  I can add
>>> another provider for my specific case but i think this is a generic case of
>>> airavata and need discussion.
>>>
>>
>> Are these scripts specific to pegasus ? If scripts are generic and can
>> provide information about running jobs its good to change existing
>> providers rather than introducing new one.
>>
>>
>>>
>>> If we want to add asyncronous submission and monitoring to current Local
>>> and SSH providers following changes are needed
>>>
>>> 1. Add flag to descriptor to identify execution pattern.
>>> 2. Change descriptor to allow users to add monitoring script path.
>>> 3. Incase we want to support job cancel, cancel script path also need to
>>> be added to descriptors also.
>>> 4. Monitoring pattern and status (to make decision) need to be
>>> configured using some interface.
>>>
>>
>> In addition to specifying specific script paths can we generate some
>> generic scripts to do job monitoring and job cancellation. For example for
>> monitoring we can check the process id status. For cancellation we can
>> execute a command like "kill -TERM <jobid>".
>>
>> Further can we make async and sync as an option in the same provider.
>> Rather than have a separate provider for Async operations.
>>
> We can specify this  WorkflowContext header and keep the synchronous
> submission as the default one.
>
> WDYT ?
>
+1
-AJ

>
> Lahiru
>
>>
>> Thanks
>> Amila
>>
>>>
>>> Adding Async Local and SSH provider for this case is another option and
>>> new descriptors can be added.
>>>
>>> Thoughts?
>>>
>>> Thanks
>>> Raminder
>>>
>>>
>>>
>>
>>
>>
>
>
> --
> System Analyst Programmer
> PTI Lab
> Indiana University
>

Mime
View raw message