oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mallder, Valerie" <Valerie.Mall...@jhuapl.edu>
Subject RE: workflow control question
Date Wed, 23 Sep 2015 15:15:50 GMT
Hi Bruce,

Thank you for your well thought out response. Now I feel bad because my question is not nearly
as big as your answer! :( I should have been a little bit more clear with my question. I am
looking for a way within OODT to prevent a second instance of a workflow from starting before
the first instance of the same workflow has finished.  I have been looking at the examples
for using WorkflowConditions to control the operation of a workflow, but there are no specific
examples that do what I would like to do. So, if anyone has an example of doing this kind
of thing, please let me know. Otherwise I will have to grow my own.  I am currently building
a custom WorkflowCondition and from within that condition class I will try to see if I can
query the workflow manager to get information about the last running workflow.

Thanks,

Val




Valerie A. Mallder
New Horizons Deputy Mission System Engineer
Johns Hopkins University/Applied Physics Laboratory

> -----Original Message-----
> From: Bruce Barkstrom [mailto:brbarkstrom@gmail.com]
> Sent: Tuesday, September 22, 2015 8:29 AM
> To: dev@oodt.apache.org
> Subject: Re: workflow control question
> 
> The usual approach to this kind of problem is to use techniques from concurrent
> programming that involve scheduling.  I'm most familiar with Ada, where there's a
> long history of work in this area.
> A classic text is
> 
> Klein, M. H., et al., 1993: A Practitioner's Handbook for Real-Time
> Analysis: Guide to Rate Monotonic Analysis for Real-Time Systems, Kluwer,
> Boston, MA
> 
> These scheduling problems are usually divided into soft problems, where the
> consequences of missing the schedule are not catastrophic and hard problems,
> where missing the schedule causes a system failure that is capable of hurting
> people.  The analysis in this reference suggests that there are two kinds of
> approaches to scheduling that can be guaranteed to work: Rate Monotonic with
> Earliest Deadline First (EDF) which allows you to take up about 70% of the
> production capacity and scheduling with homogeneous processes which allows
> you to move to nearly 100% of capacity.
> You can think of these as the difference between the traffic flow of an interstate
> highway and a railroad.  In the former, each car has some average distance
> between itself and the other vehicles, but the car can move around within that
> average distance.
> In the latter, the distance between cars is pretty close to fixed.
> 
> Two more recent works are
> 
> Burns, A. and Wellings, A., 2007: Concurrent and Real-Time Programming in Ada,
> Cambridge Univ. Press, Cambridge, UK
> 
> and
> 
> McCormick, J. W., Singhoff, F., and Hugues, J., 2011: Building Parallel,
> Embedded, and Real-Time Applications with Ada, Cambridge Univ. Press,
> Cambridge, UK
> 
> Both of these works cover various approaches to building a production scheduling
> environment.  The concerns include deadlock, resource starvation, and system
> component failures.  In cases where the system uses priorities to help derive the
> schedule, you can also have priority inversion.
> 
> The scheduling problem has a pretty large literature since it shows up not just in
> the IT environment, but also in any organization that has to deal with scheduling
> scarce resources.  You might also want to take a look at the work by Leslie
> Lamport:
> 
> <http://research.microsoft.com/en-us/um/people/lamport/pubs/pubs.html>
> 
> Lamport has an analysis tool known as TLA+ that has been used for formal
> analysis of scheduling requirements.  This tool is available online.
> You can go to the TLA Home Page
> 
> <http://research.microsoft.com/en-us/um/people/lamport/tla/tla.html>
> 
> and download it from there.
> 
> As you might expect, this kind of problem is not trivial - and even experienced
> people make design mistakes.
> 
> I don't have an easy solution to suggest to you.  To do this kind of work properly,
> you'll need to conduct an analysis based on the environment you'll be working in.
> Also, as Lamport explains, you have to worry about the basic scheduling issues -
> and then you need to deal with scheduling in the presence of unreliable
> components.  The difference between professional scheduling analysis and simple
> analysis is in whether the consequences of failure can kill people or just simply
> manually restarting the system and then figuring out what got corrupted.
> 
> Bruce B
> 
> 
> On Mon, Sep 21, 2015 at 5:27 PM, Mallder, Valerie < Valerie.Mallder@jhuapl.edu>
> wrote:
> 
> > Hi All,
> >
> > What is the easiest way to prevent an improper start of workflow?
> >
> > I have a cron job that sends an event (i.e. once an hour) to my
> > workflow manager telling workflow manager to start a workflow. But,
> > the workflow could take a long time to run depending on how many files
> > are available to be processed at that time. If the workflow takes
> > longer than an hour to complete, the cron job is going to send another
> > event to workflow manager telling it to start the workflow again. But
> > I don't want it to start the workflow again if the previous workflow
> > hasn't completed yet. It's perfectly OK for workflow manager to ignore
> > that second request to start the workflow again and just wait for the
> > next event to be sent by the cron job.
> >
> > I don't want to reinvent the wheel. Has anyone already done something
> > this?  I've looked into the workflow preconditions, and I created a
> > WorkflowStatusCondition class to use as a precondition. But, I can't
> > tell if it is possible to check the status of the first workflow
> > instance from within a WorkflowConditionInstance object in a second workflow
> instance.
> >
> > Does anyone know how I would do that?
> >
> > Val
> >
> >
> > Valerie A. Mallder
> >
> > New Horizons Deputy Mission System Engineer The Johns Hopkins
> > University/Applied Physics Laboratory
> > 11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
> > 240-228-7846 (Office) 410-504-2233 (Blackberry)
> >
> >
Mime
View raw message