airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Mattmann <mattm...@apache.org>
Subject Re: Povenance Aware Workflow Processing
Date Sat, 24 Aug 2013 02:17:18 GMT
Hey Sanjaya,

You may want to check out the Traceable Product product type policy
[1] as well as OODT-139 [2] which includes the PCS services that serve
up the information and provenance.

Cheers,
Chris

[1] 
http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/examp
les/trace/
[2] http://issues.apache.org/jira/browse/OODT-139


-----Original Message-----
From: Sanjaya Medonsa <sanjayamrt@gmail.com>
Reply-To: <dev@oodt.apache.org>
Date: Friday, August 23, 2013 6:30 PM
To: Airavata Dev <dev@airavata.apache.org>
Cc: "dev@oodt.apache.org" <dev@oodt.apache.org>
Subject: Re: Povenance Aware Workflow Processing

>Thanks Suresh! I am interested in Provenance aware workflow processing
>where OODT integration could help greatly with rich metadata available.
>I'll  go through the item 6 and let you know my understanding.
>
>Sanjaya
>
>
>On Thu, Aug 8, 2013 at 7:55 PM, Suresh Marru <smarru@apache.org> wrote:
>
>> Hi Sanjaya, Lewis,
>>
>> Appreciate your interest on this topic, this is a very good question and
>> sorry for the delay in responding, these days I feel I am always in
>>catch
>> up mode.
>>
>> Let me separate this discussion into four parts: Provenance Gathering,
>> Provenance Analyzing, Provenance Reasoning, Provenance Aware executions.
>>
>> Gathering: Airavata is instrumented with rich provenance centric
>> information model which captures each step and publishes to an event
>>bus.
>> This is encoded in a "workflow tracking" schema [1]. So the current
>>trunk
>> of Airavata publishes provenance rich information but not all is
>>guaranteed
>> to be captured in the registry.
>>
>> Analyzing: Significant work on provenance aspect related to Airavata is
>>a
>> by product of doctoral dissertation of Yogesh Simmhan, currently a
>>research
>> asst. professor at USC. His thesis [2] is a good source of reference on
>> this topic. Both the research and tools are in active development by
>>Prof.
>> Beth Plale's group at IU [3]. Theoretically speaking since both Karma
>>uses
>> the same workflow tracking schema and event bus, the tools should be
>>easily
>> integratable but did not have a motivating use case to do so.
>>
>> Reasoning: Phala is an extension to Airavata XBaya (its legacy LEAD
>> version) and Karma framework which provides context to the gathered
>> provenance. Couple of papers on this topic [4], [5].
>>
>> Executions: This is an active topic within Airavata. Current release of
>> Airavata supports Provenance aware workflow executions and is discussed
>>in
>> [6]. I think the OODT and Airavata integration will greatly benefit this
>> aspect. Airavata Registry metadata cataloging is very primitive. And
>>Rich
>> data management capabilities of OODT should nicely enhance airavata
>> provenance aware executions.
>>
>> Please let me know which of the above aspects you are interested in and
>>we
>> can discuss further.
>>
>> Suresh
>>
>> [1] -
>> 
>>https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/workflow-
>>tracking/src/main/resources/schemas/workflow_tracking_types.xsd
>> [2] - http://gradworks.umi.com/3297094.pdf
>> [3] - http://d2i.indiana.edu/provenance_karma
>> [4] -
>> http://link.springer.com/content/pdf/10.1007%2F978-3-540-85502-6_18.pdf
>> [5] -
>> 
>>http://dl.acm.org/citation.cfm?id=1590963.1591575&coll=DL&dl=GUIDE&CFID=2
>>38325269&CFTOKEN=73364358
>> [6] - http://dl.acm.org/citation.cfm?id=2359844
>>
>>
>> On Jul 22, 2013, at 10:57 AM, Lewis John Mcgibbney <
>> lewis.mcgibbney@gmail.com> wrote:
>>
>> > Hi Sanjaya,
>> > I am seeing some traffic floating around RE: Airavata OODT integration
>> but I am not clued up.
>> > Can you point me to the code? The issues? The mailing list threads?
>> > I am also extremely interested in provenance tracking of (record???)
>> data which passes through OODT. I would like to expose this as some
>>stage
>> to people who access the data.
>> > Thank you so much.
>> >
>> > On Monday, July 22, 2013, Sanjaya Medonsa <sanjayamrt@gmail.com>
>>wrote:
>> > > Hi,
>> > > I am trying to find out ways to improve Airavata provenance aware
>> workflow
>> > > processing based on rich metadata available in Apache OODT. As part
>>of
>> this
>> > > task, I am investigating how Apache Airavata is currently performing
>> > > provenance aware workflow processing. I have looked into the
>>provenance
>> > > manager implementation, where I could see lot of data related to
>> workflow
>> > > execution is saved in registry. But I could see very little of use
>>of
>> those
>> > > data. Could some one please explain the details of provenance aware
>> > > processing in Airavata and any future plans on this area?
>> > >
>> > > Best Regards,
>> > > Sanjaya
>> > >
>> >
>> > --
>> > Lewis
>> >
>>
>>



Mime
View raw message