oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bruce Barkstrom <brbarkst...@gmail.com>
Subject Re: Evaluate Suitable Scientific Workflow Language for Airavata.
Date Thu, 18 Sep 2014 17:55:07 GMT
Workflows for either producing or using scientific data
may be very simple (observer looks at a thermometer
and records a temperature using a browser form - and
used to be writes down the temperature in a log book)
to very complex (cf the article in the newest CACM:
Greengard, S., 2014: Weathering a New Era of Big
Data, CACM, 57, No. 9, 12-14 which provides some
statistics on the number of data sources and such
that weather forecasting uses).

Unfortunately, the social groupings of practitioners
are often isolated - leading to misleading views of
how simple the workflows are.  One of the critical
issues in dealing with this realistically is to figure
out when the production workflows create hierarchical
collections of data, usually because of different
versions in the production code.  Bioinformatics
has almost no reprocessing and not much in the
way of versioning.  Operational data collection
(weather and related things) has intermittent version
"glitches" - but not much reporcessing (reanalysis
projects take years).  Climate data records involving
satellite instruments can have VERY complex
collection processes and versioning.

It's also important to be careful about the disciplinary
groupings of different kinds of data users.  For example,
geological data archives can have rock cores turned
in by drilling teams in response to legal requirements
(which means prospectors may want property records
stated in English units and with local political jurisdictions),
while geologists may expect English units (if they're from
American petroleum firms), whereas academic geologists
may want MKS.

I expect choosing a language is going to be a long
and very complex social negotiation process.

Bruce B.

On Thu, Sep 18, 2014 at 1:15 PM, Mattmann, Chris A (3980) <
chris.a.mattmann@jpl.nasa.gov> wrote:

> Hi Guys,
>
> I've been interested in this too - we don't per have a specific
> OODT workflow language, but we specific workflows using XML, and
> other configuration (we are also thinking of moving to JSON for
> this).
>
> In the past I've also looked at YAWL and BPEL - both seem complex
> to me.
>
> I wonder at the end of the day if we should adopt something more
> modern like PIG or some other data flow type of language (PIG
> is really neat).
>
> Cheers,
> Chris
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398)
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: chris.a.mattmann@nasa.gov
> WWW:  http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Associate Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>
>
> -----Original Message-----
> From: Shameera Rathnayaka <shameerainfo@gmail.com>
> Reply-To: "architecture@airavata.apache.org"
> <architecture@airavata.apache.org>
> Date: Thursday, September 18, 2014 8:26 AM
> To: "architecture@airavata.apache.org" <architecture@airavata.apache.org>,
> dev <dev@airavata.apache.org>
> Subject: Evaluate Suitable Scientific Workflow Language for Airavata.
>
> >Hi All,
> >
> >As we all know Airavata has its own workflow language call XWF. When XWF
> >was introduced, main focus points are interoperability and convertibility.
> >But with years of experience it is convinced that above requirements are
> >not really useful when we come to real world use cases. And XWF is XML
> >based bulky language where we attache WSDLs and Workflow image it self.
> >But
> >with the recent changes WSDL part is being removed from XWF.
> >
> >It is worth to evaluate handy Scientific workflow languages in industry
> >and
> >find out pros and cons, at the end of this evaluation we need to come up
> >with idea how we should improve Airavata workflow language, either we can
> >improve existing XWF language, totally change to a new language available
> >in industry or write a new light weight language. Basic requirements that
> >we expect from new improvement are, high usability, flexible, light weight
> >and real time monitoring support. As you can see above requirements are
> >not
> >direct comes with workflow languages but we need workflow language which
> >help to support above requirements.
> >
> >After reading few papers and googling, initially i have come up with
> >following three existing languages,
> >1. YAWL <http://www.yawlfoundation.org/>
> >2. WS-BPEL
> >​3. SIDL
> ><http://computation.llnl.gov/casc/components/index.html#page=home>
> >
> >In my opinion SIDL is more familiar with scientific domain, Radical-SAGA
> >also uses slightly modified version of SIDL. Other than above three
> >languages we can come up with simple workflow language base on json(or
> >yaml) which support all our requirements for some extends.
> >
> >It would be grate if I can get more input regarding the $Subject form the
> >airavata community. You all are more than welcome to provide any type of
> >suggestions.
> >
> >Thanks,
> >Shameera.
> >
> >​
> >
> >--
> >Best Regards,
> >Shameera Rathnayaka.
> >
> >email: shameera AT apache.org , shameerainfo AT gmail.com
> >Blog : http://shameerarathnayaka.blogspot.com/
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message