airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lahiru Gunathilake <glah...@gmail.com>
Subject Re: Orchestration Component implementation review
Date Sun, 19 Jan 2014 22:31:50 GMT
Hi Saminda,

First thanks for reviewing the Orchestrator component.

On Fri, Jan 17, 2014 at 10:32 AM, Saminda Wijeratne <samindaw@gmail.com>wrote:

> Following are few thoughts I had during my review of the component,
>
> *Multi-threaded vs single threaded*
> If we are going to have multi-threaded job submission the implementation
> should work on handling race conditions. Essentially JobSubmitter should be
> able to "lock" an experiment request before continuing processing that
> request so that other JobSubmitters accessing the experiment requests a the
> same time would skip it.
>
+1

>
> *Orchestrator service*
> We might want to think of the possibility in future where we will be
> having multiple deployments of an Airavata service. This could particularly
> be true for SciGaP. We may have to think how some of the internal data
> structures/SPIs should be updated to accomodate such requirements in future.
>
> *Orchestrator Component configurations*
> I see alot of places where the orchestrator can have configurations. I
> think its too early finalize them, but I think we can start refactoring
> them out perhaps to the airavata-server.properties. I'm also seeing the
> orchestrator is now hardcoded to use default/admin gateway and username. I
> think it should come from the request itself.
>
I think having separate file for each component makes it clear and allow to
deploy separate. As an example if we are just deploying a light weight
orchestrator and deploy gfac as a separate JVM we do not need all the
complex configuration from airavata-server.properties.

I think to come to a conclusion about the configuration, we need to think
about the production deployment scenarios and think about an easy and clear
way to do the configuration. If we are planning to deploy some components
as separate we need to make configuration files to each and configuration.

>
> *Visibility of API functions*
> I think initialize(), shutdown() and startJobSubmitter() functions should
> not be part of the API because I don't see a scenario where the gateway
> developer would be responsible for using them. They serve a more internal
> purpose of managing the orchestrator component IMO. As Amila pointed out so
> long ago (wink) functions that do not concern outside parties should not be
> used as part of the API.
>
+1, in orchestrator-core are we going to focus on API or focus on more
internal functionality and design the interface methods and
orchestrator-service component can be use to wrap up them in to meaningful
API methods ( i was thinking may be we can wrap few mehtods of
orchestrator-core to one meaningful orchestrator-service operation).

I am not sure we need to have exactly one to one mapping in
orchestrator-core to the funcations we are going to expose to the gateway
developer.

>
> *Return values of Orchestrator API*
> IMO unless it is specifically required to do so I think the functions does
> not necessarily need to return anything other than throw exceptions when
> needed. For example the launchExperiment can simply return void if all is
> succesful and return an exception if something fails. Handling issues with
> a try catch is not only simpler but also the explanations are readily
> available for the user.
>
For testing of the component return values will be useful and when we are
wrapping up these in to real service operations (which will be exposed to
the gateway developer as an SPI) these values will be useful.
WDYT ?

>
> *Data persisted in registry*
> ExperimentRequest.getUsername() : I think we should clarify what this
> username denotes. In current API, in experiment submission we consider two
> types of users. Submission user (the user who submits the experiment to the
> Airavata Server - this is inferred by the request itself) and the execution
> user (the user who corelates to the application executions of the gateway -
> thus this user can be a different user for different gateway, eg: community
> user, gateway user).
> I think we should persist the date/time of the experiment request as well.
> Also when retrying of API functions in the case of a failure in an
> previous attempt there should be a way to not to repeat already performed
> steps or gracefully roleback and redo those required steps as necessary.
> While such actions could be transparent to the user sometimes it might make
> sense to allow user to be notified of success/failure of a retry. However
> this might mean keeping additional records at the registry level.
>
>


-- 
System Analyst Programmer
PTI Lab
Indiana University

Mime
View raw message