oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suresh Marru <sma...@apache.org>
Subject Re: Povenance Aware Workflow Processing
Date Thu, 08 Aug 2013 14:25:04 GMT
Hi Sanjaya, Lewis,

Appreciate your interest on this topic, this is a very good question and sorry for the delay
in responding, these days I feel I am always in catch up mode.

Let me separate this discussion into four parts: Provenance Gathering, Provenance Analyzing,
Provenance Reasoning, Provenance Aware executions. 

Gathering: Airavata is instrumented with rich provenance centric information model which captures
each step and publishes to an event bus. This is encoded in a "workflow tracking" schema [1].
So the current trunk of Airavata publishes provenance rich information but not all is guaranteed
to be captured in the registry. 

Analyzing: Significant work on provenance aspect related to Airavata is a by product of doctoral
dissertation of Yogesh Simmhan, currently a research asst. professor at USC. His thesis [2]
is a good source of reference on this topic. Both the research and tools are in active development
by Prof. Beth Plale's group at IU [3]. Theoretically speaking since both Karma uses the same
workflow tracking schema and event bus, the tools should be easily integratable but did not
have a motivating use case to do so.  

Reasoning: Phala is an extension to Airavata XBaya (its legacy LEAD version) and Karma framework
which provides context to the gathered provenance. Couple of papers on this topic [4], [5].

Executions: This is an active topic within Airavata. Current release of Airavata supports
Provenance aware workflow executions and is discussed in [6]. I think the OODT and Airavata
integration will greatly benefit this aspect. Airavata Registry metadata cataloging is very
primitive. And Rich data management capabilities of OODT should nicely enhance airavata provenance
aware executions.

Please let me know which of the above aspects you are interested in and we can discuss further.


Suresh

[1] - https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/workflow-tracking/src/main/resources/schemas/workflow_tracking_types.xsd
[2] - http://gradworks.umi.com/3297094.pdf
[3] - http://d2i.indiana.edu/provenance_karma
[4] - http://link.springer.com/content/pdf/10.1007%2F978-3-540-85502-6_18.pdf
[5] - http://dl.acm.org/citation.cfm?id=1590963.1591575&coll=DL&dl=GUIDE&CFID=238325269&CFTOKEN=73364358
[6] - http://dl.acm.org/citation.cfm?id=2359844


On Jul 22, 2013, at 10:57 AM, Lewis John Mcgibbney <lewis.mcgibbney@gmail.com> wrote:

> Hi Sanjaya,
> I am seeing some traffic floating around RE: Airavata OODT integration but I am not clued
up.
> Can you point me to the code? The issues? The mailing list threads?
> I am also extremely interested in provenance tracking of (record???) data which passes
through OODT. I would like to expose this as some stage to people who access the data.
> Thank you so much.
> 
> On Monday, July 22, 2013, Sanjaya Medonsa <sanjayamrt@gmail.com> wrote:
> > Hi,
> > I am trying to find out ways to improve Airavata provenance aware workflow
> > processing based on rich metadata available in Apache OODT. As part of this
> > task, I am investigating how Apache Airavata is currently performing
> > provenance aware workflow processing. I have looked into the provenance
> > manager implementation, where I could see lot of data related to workflow
> > execution is saved in registry. But I could see very little of use of those
> > data. Could some one please explain the details of provenance aware
> > processing in Airavata and any future plans on this area?
> >
> > Best Regards,
> > Sanjaya
> >
> 
> -- 
> Lewis 
> 


Mime
View raw message