taverna-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stian Soiland-Reyes <soiland-re...@cs.manchester.ac.uk>
Subject Re: Taverna and SGE
Date Wed, 26 Nov 2014 15:55:29 GMT
You can execute any command line locally and by SSH from the Tool Activity.

http://dev.mygrid.org.uk/wiki/display/tav250/Tool+service

So as SGE has a command line interface, you can use qsub and friends
from the Tool activity. Note that you can use multiple commands within
the Tool service as it is executed as a shell script - and you can
pick up arbitrary files after execution, e.g. example.o1.



What is lesser known is that the implementation of the Tool Activity
has a pluggable execution point. We have previously used this to
submit jobs to ARQ: http://dx.doi.org/10.1093/bioinformatics/btn095 --
but it should be possible to implement support for SGE in the same
way, and thus avoid having to do the qsub commands - this could become
a plugin for Taverna.

See for instance how SSH was implemented:

https://github.com/taverna/taverna-external-tool-activity/tree/maintenance/src/main/java/de/uni_luebeck/inb/knowarc/usecases/invocation/ssh


The UI implications of these are however varying a lot per grid
environment, as they all require various things to be set up for the
client tools to work, e.g. security credentials, certificates, shell
variables, etc.   Some don't have Java bindings, so you will have to
basically still execute job submit commands from within the Java
implementation.

On 26 November 2014 at 15:14, Hoeftberger, Johann
<Johann_Hoeftberger@dfci.harvard.edu> wrote:
> Hello Alan,
>
> thank you for your answer.
>
> On 11/26/2014 08:55 AM, alaninmcr wrote:
>> On 25/11/2014 16:36, Hoeftberger, Johann wrote:
>>> Can Taverna make use of a Sun Grid Engine Cluster and is there a good
>>> explanation how to do that?
>>
>> What do you mean by "make use of" ? Do you want to have a workflow be
>> run on a node in the cluster or to have individual steps within a
>> workflow run be executed on nodes in the cluster?
>
> I think both.
> The most important need for me seems to be to have the ability that
> individual steps within a workflow run on our SGE cluster transparently
> for the user which uses the workflow. So those workflow steps should be
> executed on nodes of the SGE cluster, the whole handshake for the usage
> of those steps should be done by the Taverna system itself.
>
> If that's possible, and I hope I can find a solution for that approach,
> I don't see limitation to create whole workflows running on the SGE grid
> too, except the "administration" of those workflows which should run on
> a separated local PC where the Taverna GUI is running.
>
> Or maybe later a commandline tool for the same purpose running on a
> local PC AND/OR running on the SGE Grid cluster as cronjob.
>
>
>>> I am looking for a working solution for this setting Taverna, SGE
>>> Cluster, Bioinformatic workflows.
>>> I appreciate good suggestions, let me know your ideas, please.
>>
>> I think it depends exactly what you are trying to do.
>
> My aim is to build Taverna workflows which can make use of our SGE grid
> for the computational demanding parts transparently (invisible) in the
> background. Depending on the concrete workflow this could mean that all
> workflow steps should be executed on the SGE grid.
>
> I couldn't find any SGE Taverna solution so far, it seems to me SGE
> isn't supported by Taverna!?
>
>
> Regards,
> Johann
>
>
> --
>
>
> The information in this e-mail is intended only for the person to whom it is
> addressed. If you believe this e-mail was sent to you in error and the e-mail
> contains patient information, please contact the Partners Compliance HelpLine at
> http://www.partners.org/complianceline . If the e-mail was sent to you in error
> but does not contain patient information, please contact the sender and properly
> dispose of the e-mail.
>



-- 
Stian Soiland-Reyes, myGrid team
School of Computer Science
The University of Manchester
http://soiland-reyes.com/stian/work/ http://orcid.org/0000-0001-9842-9718

Mime
View raw message