airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanjaya Medonsa <sanjaya...@gmail.com>
Subject Re: Povenance Aware Workflow Processing
Date Sat, 24 Aug 2013 01:30:51 GMT
Thanks Suresh! I am interested in Provenance aware workflow processing
where OODT integration could help greatly with rich metadata available.
I'll  go through the item 6 and let you know my understanding.

Sanjaya


On Thu, Aug 8, 2013 at 7:55 PM, Suresh Marru <smarru@apache.org> wrote:

> Hi Sanjaya, Lewis,
>
> Appreciate your interest on this topic, this is a very good question and
> sorry for the delay in responding, these days I feel I am always in catch
> up mode.
>
> Let me separate this discussion into four parts: Provenance Gathering,
> Provenance Analyzing, Provenance Reasoning, Provenance Aware executions.
>
> Gathering: Airavata is instrumented with rich provenance centric
> information model which captures each step and publishes to an event bus.
> This is encoded in a "workflow tracking" schema [1]. So the current trunk
> of Airavata publishes provenance rich information but not all is guaranteed
> to be captured in the registry.
>
> Analyzing: Significant work on provenance aspect related to Airavata is a
> by product of doctoral dissertation of Yogesh Simmhan, currently a research
> asst. professor at USC. His thesis [2] is a good source of reference on
> this topic. Both the research and tools are in active development by Prof.
> Beth Plale's group at IU [3]. Theoretically speaking since both Karma uses
> the same workflow tracking schema and event bus, the tools should be easily
> integratable but did not have a motivating use case to do so.
>
> Reasoning: Phala is an extension to Airavata XBaya (its legacy LEAD
> version) and Karma framework which provides context to the gathered
> provenance. Couple of papers on this topic [4], [5].
>
> Executions: This is an active topic within Airavata. Current release of
> Airavata supports Provenance aware workflow executions and is discussed in
> [6]. I think the OODT and Airavata integration will greatly benefit this
> aspect. Airavata Registry metadata cataloging is very primitive. And Rich
> data management capabilities of OODT should nicely enhance airavata
> provenance aware executions.
>
> Please let me know which of the above aspects you are interested in and we
> can discuss further.
>
> Suresh
>
> [1] -
> https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/workflow-tracking/src/main/resources/schemas/workflow_tracking_types.xsd
> [2] - http://gradworks.umi.com/3297094.pdf
> [3] - http://d2i.indiana.edu/provenance_karma
> [4] -
> http://link.springer.com/content/pdf/10.1007%2F978-3-540-85502-6_18.pdf
> [5] -
> http://dl.acm.org/citation.cfm?id=1590963.1591575&coll=DL&dl=GUIDE&CFID=238325269&CFTOKEN=73364358
> [6] - http://dl.acm.org/citation.cfm?id=2359844
>
>
> On Jul 22, 2013, at 10:57 AM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
> > Hi Sanjaya,
> > I am seeing some traffic floating around RE: Airavata OODT integration
> but I am not clued up.
> > Can you point me to the code? The issues? The mailing list threads?
> > I am also extremely interested in provenance tracking of (record???)
> data which passes through OODT. I would like to expose this as some stage
> to people who access the data.
> > Thank you so much.
> >
> > On Monday, July 22, 2013, Sanjaya Medonsa <sanjayamrt@gmail.com> wrote:
> > > Hi,
> > > I am trying to find out ways to improve Airavata provenance aware
> workflow
> > > processing based on rich metadata available in Apache OODT. As part of
> this
> > > task, I am investigating how Apache Airavata is currently performing
> > > provenance aware workflow processing. I have looked into the provenance
> > > manager implementation, where I could see lot of data related to
> workflow
> > > execution is saved in registry. But I could see very little of use of
> those
> > > data. Could some one please explain the details of provenance aware
> > > processing in Airavata and any future plans on this area?
> > >
> > > Best Regards,
> > > Sanjaya
> > >
> >
> > --
> > Lewis
> >
>
>

Mime
View raw message