taverna-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stian Soiland-Reyes <st...@apache.org>
Subject Re: Common workflow language
Date Thu, 10 Dec 2015 09:35:47 GMT
 I've signed up for the CWL hackathon, but I hope others would join!

I find CWL interesting as it has focused on the tool descriptions, and
packaging them using Docker. This is a pragmatic approach that means
lots of issues on reproducibility, installation and repurposability is
pretty much solved.

It does however means it is hard to use non-local tools like REST
services, as you would need an equivalent command line tool.  Message
parsing and shims are trickier, but on the plus side you could use
your favourite scripting language (as long as it works on Linux :)).

Docker is also an important part of http://bioboxes.org/

CWL Tool descriptions are annotated with the EDAM ontology, which
includes many ELIXIR folks. ELIXIR is building a tool registry.

Now I think it would be nice if we could also use such a tool registry
and descriptions from Taverna.

CWL focuses on data as files, in a way that is very similar to the
current Tool service.

I think we could update the Tool Service to handle "CWL tools" or at
least handle docker images.  Of course you could do this today with
just having a command line tool that is "docker run blablabla".

I would like to start work on a CWL reader for SCUFL2, which would
create such Tool instances in the first place.

Representing any SCUFL2/Taverna workflow in CWL can be tricky, as you
would need to extend the tool type to have REST, WSDL, etc/. This is
an extension point in CWL though - and so it would perhaps raise the
question of why use CWL with custom extensions if nobody else can run
such the Taverna services. Still, the workflow structure can be
repurposed, with a different activity inserted in other CWL engines.
There's a danger here of "standards mutation" - like we saw a decade
ago with WSDL being modified beyond recognition in Globus web services
and WSRF.

Another approach could be to create Docker images that can execute
Taverna activities (e.g. using the wsdl-generic command line). This
would be more of an interesting research angle, perhaps, as it means
we would be more freely able to try to run a Taverna workflow with an
alternate CWL workflow engine.

Taverna workflows allow various control mechanisms like Retry,
Failover, Parallelism, inplicit iterations, dot/cross product - which
could be tricky to represent in CWL. I think only a subset of Taverna
workflows could be run as CWL - without again adding customizations -
kind of like a CWL profile.

I think one aspect is that CWL is also a focus opportunity for Taverna
- coordinating command line tools is simpler than trying to integrate
"everything". I think also this is more of the growing space we find
Taverna in now, rather than 8 years ago when Taverna aimed to be
bioinformatician's best friend and we added lots of GUI and activity
plugins for various bioinformatics-centric services. (however
AstroTaverna is one landscape which is still open to be claimed in
such a way.).  CWL is a bit more on a "hacker level" - which I think
we can be more comfortable with catering for.

For CWL, Taverna is a valuable proposition as we know how to handle
workflow provenance, we can easily add preservation aspects (e.g.
packaging all the docker images used in a run).  The Taverna Workbench
has a pluggable way to add service descriptions, so we could connect
to CWL tool registries and turn Taverna into a "CWL editor".

On 4 December 2015 at 17:05, Alan Williams <alaninmcr@googlemail.com> wrote:
> On 04-Dec-15 15:28, Ian Dunlop wrote:
>> Hash: SHA256
>> Hello,
>> Does anyone have views on how Apache Taverna fits into the goals of
>> the Common Workflow Language project
>> https://github.com/common-workflow-language/common-workflow-language ?
>> Should taverna adopt CWL instead of SCUFL2. Are there things that
>> SCUFL2 or the current taverna engine offers that CWL does not?
> Stian is probably the best person to answer this as he has been quite
> involved with CWL. To the best of my knowledge, CWL started off very similar
> to SCUFL2 but diverged.
> There is a CWL/Taverna/Galaxy hackathon planned for May 2016 in Paris. I
> think (hope) I sent around the Doodle poll for it.
>> Cheers,
>> Ian
> Alan

Stian Soiland-Reyes
Apache Taverna (incubating), Apache Commons RDF (incubating)

View raw message