airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chathuri Wimalasena <kamalas...@gmail.com>
Subject Re: Orchestration Component implementation review
Date Fri, 17 Jan 2014 16:40:25 GMT
Orchestrator table has only the current state (updated state). Previous
statuses should be saved in the GFac_Job_Status table.

Regards,
Chathuri


On Fri, Jan 17, 2014 at 11:25 AM, Sachith Withana <swsachith@gmail.com>wrote:

> Thanks Saminda for this informative review.
>
> In the case of the multi-threaded vs Single Threaded, where should we have
> the synchronization enforced?
> To my knowledge, the NewJobWorkers( Getting new Jobs and submitting them)
> and the HangedJobWorkers are accessing the Orchestrator table to select the
> new and hanged jobs.
> Right now, the NewJobworkers are getting all the accepted jobs at once.
> it's not focussed on one experiment.
>
> We need to reflect the changes in the Gfac job Statuses in the
> Orchestrator table as well. So every time the status of a job change
> through the Gfac, it will be accessing the Orchestrator table as well. (
> I've sent an email previously describing the scenario)
>
>
>
>
>
> On Fri, Jan 17, 2014 at 10:32 AM, Saminda Wijeratne <samindaw@gmail.com>wrote:
>
>> Following are few thoughts I had during my review of the component,
>>
>> *Multi-threaded vs single threaded*
>> If we are going to have multi-threaded job submission the implementation
>> should work on handling race conditions. Essentially JobSubmitter should be
>> able to "lock" an experiment request before continuing processing that
>> request so that other JobSubmitters accessing the experiment requests a the
>> same time would skip it.
>>
>> *Orchestrator service*
>> We might want to think of the possibility in future where we will be
>> having multiple deployments of an Airavata service. This could particularly
>> be true for SciGaP. We may have to think how some of the internal data
>> structures/SPIs should be updated to accomodate such requirements in future.
>>
>> *Orchestrator Component configurations*
>> I see alot of places where the orchestrator can have configurations. I
>> think its too early finalize them, but I think we can start refactoring
>> them out perhaps to the airavata-server.properties. I'm also seeing the
>> orchestrator is now hardcoded to use default/admin gateway and username. I
>> think it should come from the request itself.
>>
>> *Visibility of API functions*
>> I think initialize(), shutdown() and startJobSubmitter() functions should
>> not be part of the API because I don't see a scenario where the gateway
>> developer would be responsible for using them. They serve a more internal
>> purpose of managing the orchestrator component IMO. As Amila pointed out so
>> long ago (wink) functions that do not concern outside parties should not be
>> used as part of the API.
>>
>> *Return values of Orchestrator API*
>> IMO unless it is specifically required to do so I think the functions
>> does not necessarily need to return anything other than throw exceptions
>> when needed. For example the launchExperiment can simply return void if all
>> is succesful and return an exception if something fails. Handling issues
>> with a try catch is not only simpler but also the explanations are readily
>> available for the user.
>>
>> *Data persisted in registry*
>> ExperimentRequest.getUsername() : I think we should clarify what this
>> username denotes. In current API, in experiment submission we consider two
>> types of users. Submission user (the user who submits the experiment to the
>> Airavata Server - this is inferred by the request itself) and the execution
>> user (the user who corelates to the application executions of the gateway -
>> thus this user can be a different user for different gateway, eg: community
>> user, gateway user).
>> I think we should persist the date/time of the experiment request as
>> well.
>> Also when retrying of API functions in the case of a failure in an
>> previous attempt there should be a way to not to repeat already performed
>> steps or gracefully roleback and redo those required steps as necessary.
>> While such actions could be transparent to the user sometimes it might make
>> sense to allow user to be notified of success/failure of a retry. However
>> this might mean keeping additional records at the registry level.
>>
>>
>
>
> --
> Thanks,
> Sachith Withana
>
>

Mime
View raw message