oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cayanan, Michael D (388J)" <michael.d.caya...@jpl.nasa.gov>
Subject Re: Workflow2 - State Persistence
Date Wed, 13 Mar 2013 17:40:40 GMT
Hey Chris,

Sounds awesome. +1 to the additional features. I'm pretty sure I'll be able to help out as
we go further along in developing this pipeline service for AMMOS. They eventually would like
this feature and are asking me how much effort is required to do this.

I think from their end, what they want, from what I understand, is the following use-case

1) Run a command that will take a snapshot of the Workflow.
2) Bring the system down. Don't wait for anything to finish.
3) Upon restart, the Workflow should resume from where you last took the snapshot.

The AMMOS folks are worried that consistently doing state persistence will be a performance
hog and so by having the ability to save the state on demand, they'll be able to have things
run more efficiently.

Hopefully we can tag up soon to go over this so that I can let them know how much effort this
implementation would take.


From: <Mattmann>, Chris Mattmann <Chris.A.Mattmann@jpl.nasa.gov<mailto:Chris.A.Mattmann@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<mailto:user@oodt.apache.org>" <user@oodt.apache.org<mailto:user@oodt.apache.org>>
Date: Tuesday, March 12, 2013 8:32 AM
To: "user@oodt.apache.org<mailto:user@oodt.apache.org>" <user@oodt.apache.org<mailto:user@oodt.apache.org>>
Subject: Re: Workflow2 - State Persistence

Hey Mike,

Workflow information (state, etc.) is persisted to the WorkflowInstanceRepository. In Wengine/Workflow2
support in trunk it makes heavy use of this, but the support is evolving.

I think for 0.6 I'll add some functionality and features (and would appreciate any help) to
allow users to:

  1.  Specify in workflow.properties whether or not prior not completed (or not "finished"
category state) workflows should be cleared out and/or should be restarted and run on startup.
Maybe a property like org.apache.oodt.cas.workflow.engine.unfinishedWorkflows.complete=true
or false
  2.  Finish support in WorkflowInstanceRepository for persisting all new Wengine state information

Those are the 2 items that should be done for 0.6 to support this feature. It's on the roadmap
just not fully done yet.


From: <Cayanan>, "Michael D (388J)" <michael.d.cayanan@jpl.nasa.gov<mailto:michael.d.cayanan@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<mailto:user@oodt.apache.org>" <user@oodt.apache.org<mailto:user@oodt.apache.org>>
Date: Thursday, March 7, 2013 8:41 AM
To: "user@oodt.apache.org<mailto:user@oodt.apache.org>" <user@oodt.apache.org<mailto:user@oodt.apache.org>>
Subject: Workflow2 - State Persistence

Hi Chris and/or Brian,

I'm currently on a project where PDS and AMMOS are joining forces to create a pipeline service
and we'd like to use the latest Workflow2 to do that. They asked a question at last week's
meeting regarding if the latest and greatest OODT Workflow will be able to do state persistence.
I can let Paul Ramirez chime in any further details since he was at the meeting as well, but
can the latest and greatest Workflow do that? And if so, under which engine? I'm currently
using the following engine:

workflow.engine.factory = org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory



View raw message