commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paulo Gaspar" <paulo.gas...@krankikom.de>
Subject RE: Workflow & Turbine design questions
Date Sat, 01 Dec 2001 16:46:40 GMT
On persistency, it makes much more sense to me to do it by name, by
previously registering your tasks in some kind of registry (JNDI?).

IMO there are 2 different use cases:
  1 - Simple and light;
  2 - Robust workhorse.

Going further:
  1 - is when you just want to have scheduling inside a very simple
      application by registering Runnable objects (possibly even
      inner objects) to run at specific times or time intervals.
      This is handy even to things like pinging a service or cleaning
      up a pool of objects.

  2 - is when you have a big server application that has run important
      batches of jobs, uses J2EE and all and can never let anything
      fail.

Users in case 1 possibly do not want to care about all the "J2EE crap"
and do not want to be loaded with persistency concerns and so.

Users in case 2 can be served by building a wrapper around the
components of level 1. They take full advantage of the basic component
and extend it to their own needs, which can vary a lot in complexity.

It probably makes sense to make a level 1 package and a level 2
package.

A simple level 2 package would just support running JNDI registered
tasks and a simple task to send JMS messages. It would always persist
its state and make sure not to run things twice due to failure.

This level 2 package is flexible enough to cover most "level 2" needs,
would supply must of the hard to do stuff and be easy to customize for
other needs.

At the same time, persisting JNDI names is much easier and trouble
free than persisting objects, with all the attached problems, like
implementation changes between runs and so.


What do you think about this?


have fun,
Paulo Gaspar


> -----Original Message-----
> From: Erik Worth [mailto:eworth@altoweb.com]
> Sent: Friday, November 30, 2001 8:26 PM
> To: Jakarta Commons Developers List
> Subject: RE: Workflow & Turbine design questions
>
>
> Hi Gang,
> The tricky thing with building a scheduler suitable for workflow
> is the need
> to schedule tasks in the future (i.e. do this task 10 days from
> now at 2:35
> PM).  When scheduling future tasks, one needs to persist the
> schedule.  This
> type of service is also difficult to support in a clustered or fault
> tolerant environment because it is relatively difficult to keep multiple
> instances of the same scheduler in sync so that when the main one
> (you only
> want one of these scheduling tasks at a time or you get duplicate work
> scheduled) goes down, the other one automatically picks up where the other
> left off.
>
> I am not trying to dampen anyone's enthusiasm with regards to building one
> of these for the open source community, but it would be nice if such an
> endeavor was taken up, it would address these types of issues.  I built a
> CORBA-based implementation of such a scheduler a number of years back that
> handled the persistence case, but when we considered the level of
> effort for
> a fault tolerant version, we decided to live with the single point of
> failure.
>
> One aspect of the approach I took worked out rather well.  When
> scheduling a
> task, I passed a command object along with the schedule time and recipient
> (and periodicity if any).  If this were implemented using JMS as the
> transport, one could pass a serializable command object (an object
> implementing an interface that specifies an execute method), a JMS
> destination (as the recipient), the schedule time, and the
> periodicity using
> an JMS ObjectMessage payload.  The scheduler (in this case a stand-alone
> daemon type application) keeps a time ordered list of tasks and when a
> task's time comes due, sends the command object to the designated
> destination and removes the task if is not a periodic task.  If Message
> Driven Beans are listening on the destination queues, the threading is
> already taken care of and the listener simply invokes the execute
> method on
> the command object (simple JMS listeners can also execute tasks, but they
> need to manage their own thread pool).  The execute method is
> implemented to
> perform a task or invoke on something that performs the task.
>
> Using this model, a client is able to schedule some other service
> to perform
> a task at a specific time in the future (or periodically starting at some
> time in the future).  It decouples the schedule maintenance from the task
> execution, and it has a hope of being made fault tolerant.
>
> Food for thought :-)
> -- Erik.
>
>
> -----Original Message-----
> From: James Strachan [mailto:james_strachan@yahoo.co.uk]
> Sent: Friday, November 30, 2001 2:42 AM
> To: Jakarta Commons Developers List
> Subject: Re: Workflow & Turbine design questions
>
>
> +1.
>
> I've wanted a scheduler component for some time - up until now everything
> I've seen has been integrated into larger frameworks or has been payware.
> Maybe a combination of a general ThreadPool that asynchronous tasks can be
> dispatched to (probably using the Runnable interface to represent
> the tasks)
> with a Scheduler capable of dispatching tasks at fixed or repeated
> intervals.
>
> The Timer and TimerTask in JDK1.3 are a pretty good start for a
> Scheduler if
> used with ThreadPools for the heavy-lifting. Though
> Timer/TimerTask is only
> available on 1.3. Its a real shame we're not allowed to
> redistribute classes
> from the newer JDKs so we can use them on older JDKs like 1.2. (Ditto for
> other stuff in 1.4).
>
> James
> ----- Original Message -----
> From: "Craig R. McClanahan" <craigmcc@apache.org>
> To: "Jakarta Commons Developers List" <commons-dev@jakarta.apache.org>;
> <brettgullan@mac.com>
> Sent: Friday, November 30, 2001 2:32 AM
> Subject: RE: Workflow & Turbine design questions
>
>
> > One way to approach these issues (and would even fit into the Commons
> > charter :-) would be to conceive of a Commons package to do
> background job
> > scheduling and execution (what legacy OSs called "batch job queues")
> > within a JVM.  Executing a particular workflow script would be
> only one of
> > the interesting things you could do -- ideally the definition of a "job"
> > would be something like "call method X on object Y" so it could be fully
> > generalized.
> >
> > The "jobs" package could be configured with limits on how many
> background
> > threads it could run, support ways to schedule jobs repeatedly, and all
> > the other stuff you can do with "cron" type utilities, if we wanted.  It
> > could be embedded in any sort of long-running JVM (such as in a
> webapp on
> > a servlet container), as well as stand-alone background jobs.
> >
> > This is a bigger task than just something that can run Workflow
> scripts in
> > background threads (which isn't terribly difficult to hard code in your
> > application by starting a new thread and executing the Activity there),
> > but it would be a generally useful package.  What do you think?
> >
> > Craig
> >
> >
> > On Thu, 29 Nov 2001, Brett Gullan wrote:
> >
> > > Date: Thu, 29 Nov 2001 09:24:55 +0800
> > > From: Brett Gullan <brettgullan@mac.com>
> > > Reply-To: Jakarta Commons Developers List
> <commons-dev@jakarta.apache.org>,
> > >      brettgullan@mac.com
> > > To: 'Jakarta Commons Developers List' <commons-dev@jakarta.apache.org>
> > > Subject: RE: Workflow & Turbine design questions
> > >
> > > Almost...
> > >
> > > > #1. User starts workflow process by performing action.
> > > Let's say User A starts workflow process, by for instance, copying a
> > > file from his/her desktop to a directory on a local/remote
> server (this
> > > can/will be achieved by a variety of methods/protocols).
> > >
> > > > #2. Workflow performs tasks that may take time.
> > > Yes
> > >
> > > > #3. User should not be held up waiting for a response from
> the server
> > > by
> > > background workflow tasks.
> > > Yes. However, User A may never even use the Web UI. User B on
> the other
> > > hand may be watching the queue/progress monitor for regular updates.
> > >
> > > The web UI is intended to be a monitoring and configuration
> tool for the
> > > underlying workflow engine. For the most part, general users would
> > > submit files (by desktop drag-n-drop, FTP, web-form, etc...) then use
> > > the web UI to monitor progress or change the queue priority (if
> > > authorised).
> > > Administrative users would have access to web forms that configure the
> > > workflow processing activity steps and the 'hot' folders that feed the
> > > processing queue.
> > > The application is intended to support deployment on a
> 'headless' server
> > > where all interaction is via a web UI.
> > > I hope that makes more sense?
> > > Regards,
> > > Brett
> > > --
> > > Brett Gullan
> > > brettgullan@mac.com
> > >
> > >
> > >
> > > --
> > > To unsubscribe, e-mail:
> <mailto:commons-dev-unsubscribe@jakarta.apache.org>
> > > For additional commands, e-mail:
> <mailto:commons-dev-help@jakarta.apache.org>
> > >
> > >
> >
> >
> > --
> > To unsubscribe, e-mail:
> <mailto:commons-dev-unsubscribe@jakarta.apache.org>
> > For additional commands, e-mail:
> <mailto:commons-dev-help@jakarta.apache.org>
>
>
> _________________________________________________________
> Do You Yahoo!?
> Get your free @yahoo.com address at http://mail.yahoo.com
>
>
> --
> To unsubscribe, e-mail:
> <mailto:commons-dev-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail:
> <mailto:commons-dev-help@jakarta.apache.org>
>
> --
> To unsubscribe, e-mail:
<mailto:commons-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:commons-dev-help@jakarta.apache.org>


--
To unsubscribe, e-mail:   <mailto:commons-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:commons-dev-help@jakarta.apache.org>


Mime
View raw message