airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanjaya Medonsa <sanjaya...@gmail.com>
Subject Re: Povenance Aware Workflow Processing
Date Tue, 23 Jul 2013 05:08:59 GMT
Hi,
As per my understanding, in simple terms, provenance aware workflow
processing means if output is available from previous execution, then that
particular task doesn't require to execute again as output is already
available. Then that output can be used for rest of the workflow execution.
Provenance aware workflow processing  avoids re executing already executed
tasks, thus improving overall performance. Please correct if my
understanding is wrong.

When it comes to Airavata, what is the best way to identify previously
executed tasks. Is it from input and the service name ? Please let me know
your thoughts. Here I am trying to understand best way to use the data
available in Airavata Registry and the OODT metadata catalog. As part of
the OODT-integration, output of task execution ingest into OODT file
manager server together with corresponding meta data. Basically I need to
understand what sort of Metadata that needs to be stored together with the
output. At the moment I am planning to store service name and workflow
instance id together with other metadata extracted based on output by OODT
metadata extractors. I guess I may need to store input as well.
Please let me know your valuable thoughts, it would be really helpful.

Best Regards,
Sanjaya


On Mon, Jul 22, 2013 at 11:13 PM, Sanjaya Medonsa <sanjayamrt@gmail.com>wrote:

> Hi Lewis,
>      As you mentioned, RE: Airavata OODT integration is the main email
> thread related to my work. Still code has not been checked in. Earlier I
> have published a review request for my code changes. I have got some
> feedback from Chris and I am planning to publish a new review request
> during next week with the updates. In short, I am using OODT task wrapper
> capabilities to improve Airavata workflow execution. Basically there are
> two main tasks.
>    1. Stage input file from OODT file manager server as input for Airavata
> workflow execution
>    2. Ingest generated output of a Airavata workflow execution to OODT
> file manager server with metadata
>
> Improving provenance aware workflow processing is my next task. Hope this
> helps you. Please let me know if you need more details.
>
> Best Regards,
> Sanjaya
>
>
> On Mon, Jul 22, 2013 at 8:27 PM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
>> Hi Sanjaya,
>> I am seeing some traffic floating around RE: Airavata OODT integration
>> but I am not clued up.
>> Can you point me to the code? The issues? The mailing list threads?
>> I am also extremely interested in provenance tracking of (record???) data
>> which passes through OODT. I would like to expose this as some stage to
>> people who access the data.
>> Thank you so much.
>>
>>
>> On Monday, July 22, 2013, Sanjaya Medonsa <sanjayamrt@gmail.com> wrote:
>> > Hi,
>> > I am trying to find out ways to improve Airavata provenance aware
>> workflow
>> > processing based on rich metadata available in Apache OODT. As part of
>> this
>> > task, I am investigating how Apache Airavata is currently performing
>> > provenance aware workflow processing. I have looked into the provenance
>> > manager implementation, where I could see lot of data related to
>> workflow
>> > execution is saved in registry. But I could see very little of use of
>> those
>> > data. Could some one please explain the details of provenance aware
>> > processing in Airavata and any future plans on this area?
>> >
>> > Best Regards,
>> > Sanjaya
>> >
>>
>> --
>> *Lewis*
>>
>>
>

Mime
View raw message