oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (388J)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: Workflow Question
Date Wed, 07 Mar 2012 06:26:40 GMT
Hey Bruce,

On Mar 6, 2012, at 9:29 AM, Bruce Barkstrom wrote:

> This question is a bit like asking the classic ftp software
> when has the file finished transferring.  The answer seems to me
> to be similar to the transaction protocols that produce
> reliable ACID behavior.  Perhaps it could be one of those
> cases that could be handled by carefully crafted exception
> handlers: system timed out after [limit], transaction rolled
> back with message to log, sender, and receiver.

Agree! In pre 0.4 versions of OODT, this was supported by the 
sense that you could always use the "resumeWorkflowInst" feature
to "unpause" a workflow waiting in a condition forever. 

In 0.4-SNAPSHOT (and the eventual release), workflow conditions now
support configurable timeouts, see OODT-215 [1] in general and more
specifically OODT-

Cheers,
Chris

[1] https://issues.apache.org/jira/browse/OODT-215
[2] https://issues.apache.org/jira/browse/OODT-207

> 
> Bruce B.
> 
> On Tue, Mar 6, 2012 at 11:31 AM, Keith Cummings <kcumming@nrao.edu> wrote:
>> I have the same question Michael is asking (#1 below).  If a precondition
>> never passes, I do not want that workflow-task to indefinitely poll on that
>> precondition.  At some point (preferably a configurable amount of time) it
>> should give up and fail (halt) the workflow.  Or is there another way to
>> address this Use Case?
>> -Keith Cummings
>> 
>> 
>> Cayanan, Michael D (388J) wrote:
>> 
>> Hi Chris,
>> 
>> On 3/2/12 9:36 PM, "Mattmann, Chris A (388J)"
>> <chris.a.mattmann@jpl.nasa.gov> wrote:
>> 
>> 
>> 
>> Hi Mike,
>> 
>> On Mar 2, 2012, at 9:37 AM, Cayanan, Michael D (388J) wrote:
>> 
>> 
>> 
>> Hi all,
>> 
>> In the workflow.properties file, I see that I can configure the polling
>> time if a pre condition fails via the following property:
>> 
>> org.apache.oodt.cas.workflow.engine.preConditionWaitTime
>> 
>> I have a few questions:
>> 
>> 1) Is this polling infinite?
>> 
>> 
>> What do you mean by "infinite"?
>> 
>> 
>> 
>> If so, would you have to shut the workflow server down in order to stop
>> this infinite polling? Or can you shut it down via the workflow manager
>> client tool?
>> 
>> 
>> I'm not sure I follow you? The workflow manager's
>> ThreadPoolWorkflowEngine needs to evaluate preconditions, on a per
>> workflow-instance basis.
>> The only way to shut that down, is to shut down the workflow engine, and
>> thus implicitly the WM.
>> 
>> 
>> In other words, if the pre condition continually fails, will the workflow
>> ever timeout where the engine shuts down automatically?
>> 
>> 
>> 
>> 
>> 2) Is there a way to configure the pre condition wait time through the
>> PGE config file? For example, if I have 2 PGEs where I want each PGE to
>> have different pre condition wait times, is this possible?
>> 
>> 
>> Can you tell me a specific use case or reason that you would want to have
>> different condition check times per PGE?
>> 
>> In short, this can be supported by extending the
>> ThreadPoolWorkflowEngine, and then overriding or mapping the condition
>> wait
>> time to whatever is configured for the PGE. I'd severely caution against
>> doing so, b/c you're really giving the developer or
>> workflow person an overt amount of control over an aspect of the system
>> that will affect (perceived) performance.
>> 
>> 
>> This was more of a curiosity question than anything. Personally, I don't
>> see the need to have a configurable pre condition wait time. In SMAP
>> world, there will be a case where a workflow requires two inputs, where
>> the inputs are coming from the outputs of 2 different workflows.
>> Furthermore, 1 of these inputs isn't expected to arrive like 10 hours
>> later after the first input arrives. The assumption here is that 1 input
>> will trigger the workflow and thus the pre condition would be to check for
>> the existence for that other input. So, if you have your precondition wait
>> time to some small interval like 30 seconds, would system performance be
>> affected in this case knowing that the precondition isn't expected to be
>> satisfied until 10 hours later?
>> 
>> Hope this makes sense.
>> 
>> Thanks again for your help!
>> 
>> -Mike
>> 
>> 
>> 
>> Cheers,
>> Chris
>> 
>> 
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW:   http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> 
>> 
>> 
>> 


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


Mime
View raw message