taverna-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Dunlop <ian.dun...@manchester.ac.uk>
Subject Re: Common workflow language
Date Fri, 11 Dec 2015 14:02:20 GMT

I know this is a crazy suggestion, but......are there some issues that
we can add to JIRA around CWL or jbatch. Otherwise it will be lost in
the email thread.

* CWL reader for SCUFL2
* Docker image for taverna activities (docker compose based?)
* Integrate with CWL tool registries (which I guess means getting the
workbench working!)



On 10/12/2015 09:35, Stian Soiland-Reyes wrote:
>  I've signed up for the CWL hackathon, but I hope others would join!
> I find CWL interesting as it has focused on the tool descriptions, and
> packaging them using Docker. This is a pragmatic approach that means
> lots of issues on reproducibility, installation and repurposability is
> pretty much solved.
> It does however means it is hard to use non-local tools like REST
> services, as you would need an equivalent command line tool.  Message
> parsing and shims are trickier, but on the plus side you could use
> your favourite scripting language (as long as it works on Linux :)).
> Docker is also an important part of http://bioboxes.org/
> CWL Tool descriptions are annotated with the EDAM ontology, which
> includes many ELIXIR folks. ELIXIR is building a tool registry.
> Now I think it would be nice if we could also use such a tool registry
> and descriptions from Taverna.
> CWL focuses on data as files, in a way that is very similar to the
> current Tool service.
> I think we could update the Tool Service to handle "CWL tools" or at
> least handle docker images.  Of course you could do this today with
> just having a command line tool that is "docker run blablabla".
> I would like to start work on a CWL reader for SCUFL2, which would
> create such Tool instances in the first place.
> Representing any SCUFL2/Taverna workflow in CWL can be tricky, as you
> would need to extend the tool type to have REST, WSDL, etc/. This is
> an extension point in CWL though - and so it would perhaps raise the
> question of why use CWL with custom extensions if nobody else can run
> such the Taverna services. Still, the workflow structure can be
> repurposed, with a different activity inserted in other CWL engines.
> There's a danger here of "standards mutation" - like we saw a decade
> ago with WSDL being modified beyond recognition in Globus web services
> and WSRF.
> Another approach could be to create Docker images that can execute
> Taverna activities (e.g. using the wsdl-generic command line). This
> would be more of an interesting research angle, perhaps, as it means
> we would be more freely able to try to run a Taverna workflow with an
> alternate CWL workflow engine.
> Taverna workflows allow various control mechanisms like Retry,
> Failover, Parallelism, inplicit iterations, dot/cross product - which
> could be tricky to represent in CWL. I think only a subset of Taverna
> workflows could be run as CWL - without again adding customizations -
> kind of like a CWL profile.
> I think one aspect is that CWL is also a focus opportunity for Taverna
> - coordinating command line tools is simpler than trying to integrate
> "everything". I think also this is more of the growing space we find
> Taverna in now, rather than 8 years ago when Taverna aimed to be
> bioinformatician's best friend and we added lots of GUI and activity
> plugins for various bioinformatics-centric services. (however
> AstroTaverna is one landscape which is still open to be claimed in
> such a way.).  CWL is a bit more on a "hacker level" - which I think
> we can be more comfortable with catering for.
> For CWL, Taverna is a valuable proposition as we know how to handle
> workflow provenance, we can easily add preservation aspects (e.g.
> packaging all the docker images used in a run).  The Taverna Workbench
> has a pluggable way to add service descriptions, so we could connect
> to CWL tool registries and turn Taverna into a "CWL editor".
> On 4 December 2015 at 17:05, Alan Williams <alaninmcr@googlemail.com> wrote:
>> On 04-Dec-15 15:28, Ian Dunlop wrote:
>>> Hash: SHA256
>>> Hello,
>>> Does anyone have views on how Apache Taverna fits into the goals of
>>> the Common Workflow Language project
>>> https://github.com/common-workflow-language/common-workflow-language ?
>>> Should taverna adopt CWL instead of SCUFL2. Are there things that
>>> SCUFL2 or the current taverna engine offers that CWL does not?
>> Stian is probably the best person to answer this as he has been quite
>> involved with CWL. To the best of my knowledge, CWL started off very similar
>> to SCUFL2 but diverged.
>> There is a CWL/Taverna/Galaxy hackathon planned for May 2016 in Paris. I
>> think (hope) I sent around the Doodle poll for it.
>>> Cheers,
>>> Ian
>> Alan

Ian Dunlop, eScience Lab
School of Computer Science
The University of Manchester

View raw message