cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Giacomo Pati <giac...@apache.org>
Subject Re: CommandManager issues [was Re: Releasing 2.1.1?]
Date Sun, 31 Aug 2003 18:38:31 GMT
On Sun, 31 Aug 2003, Bruno Dumon wrote:

> On Sat, 2003-08-30 at 17:24, Bruno Dumon wrote:
> > On Sat, 2003-08-30 at 15:20, Giacomo Pati wrote:
> <snip/>
> > > I like the way how the Cornerstone Scheduler recently integrated by
> > > Carsten more than the CommandManager way because of its
> > > componentisation. Still, I like to see a Scheduler as a single component
> > > definition in the xconf file. Today we need at least three component
> > > definitions (Scheduler, ThreadManager, TimeScheduler) as well as one for
> > > each Target. I think this can be made much easier, like:
> > >
> > >   <scheduler logger="scheduler">
> > >     <triggers>
> > >       <trigger name="mytarget"
> > >                class="my.comp.MyTarget"
> > >                logger="mytarget">
> > >         <timed type="crontab">
> > >           <month> -1 </month>
> > >           <weekday> SUN </weekday>
> > >           <day> * </day>
> > >           <hour> 9-17/2 </hour>
> > >           <minute> */5 </minute>
> > >         </timed>
> > >       ...
> > >     </triggers>
> > >   </scheduler>
> > >
> > > In the above, triggers are handled as Avalon components and the
> > > scheduler takes care of instatiation and the lifecycle stuff.
> > >
> > > Unfortunately, the Cornerstone Scheduler toolkit doensn't offer a full
> > > crontab like specification of defining trigger point in time like I've
> > > used above (and I'd be happy to have).
> > >
> > > My suggestion (which I can implement if accepted) is to have a
> > > Scheduler that is able to do the above by use of the CommandManager
> > > given someone can fix Doug Leas concurrent utilities with the patch
> > > from Bruno.
> >
> > great.
>
> Or maybe not so great. I'm not sure the CommandManager is well suited
> for a general purpose scheduler.
>
> What follows now should be read with a 'AFAIU' disclaimer:
>
> All commands added to the CommandManager are executed sequentially and
> on one thread. So if there is one command that takes somewhat longer to
> execute, it will block the execution of other commands. This may be fine
> for management-tasks which should be executed repeatedly (but not at
> fixed times), but for a scheduler I would expect a command to be
> executed at the configured time.

This is a matter of configuring a ThreadManager (which admittedly
today the one created in the Cocoon class isn't configurable at all).

> This could be solved by creating a different CommandManager for each
> scheduled task, since multiple CommandManagers can be executed in

If you have a ThreadManager with as many threads as needed you're set.
You only need ONE CommandManager, AFAIU. Looking at the ThreadManager
code I quite don't understand what it is doing as it could use the
Executor from Doug Leas package more efficently.

> parallel. But then you'd still want to use a different ThreadManager
> than the TPCThreadManager, since that will block when all threads are in
> use.

Or at least a configurable one.

> The result will however be rather overkill IMHO: even commands that
> need to be executed only weekly, will be checked every 100 ms (=
> sleeptime parameter) to see if they need to be executed.

Well, IMHO 100ms is far too fast even for the ContinuationManager. For
the Scheduler a granularity of one minute is enough.

> Conclusion: while this event-based infrastructure is quite cool, I'm not
> convinced yet it is good for scheduling-purposes.

First, if you look at the Cornerstone Scheduler you'll see that the
ability to define repeated tasks are somehow limited. Business apps can
easily request administration tasks to be run twice a day on work days.
To define this with the Cornerstone Scheduler you'll need to add 10
'cron' entries to it. This is why I'd like to extend it using the unix
cron direction to define ranges and list of timing aspects. But then
you're faced with the problem of defining sleep time for efficient use
(which you need to be able to cooperate in the Cornerstone Scheduler
machinery).

The unix cron daemon runs every minute to check if there are any jobs to
run. It checks initially creating bitsets for the minute, hour, day,
month, and weekday definition for each task. This makes it easy to
check whether a task has to be launched. I still haven't found an
algorithm (and I think there isn't an easy one available) to determine
the sleep time until the next task is due. Thus, a check every minute
isn't much but still 1440 times a day.

The other thing to respect if you'd like to go as efficent as possible
is the determination of Threads you'll need to run each cron task at the
time specified. But this needs knowledge about how much elapsed time a
task need to run to decide if you need one or two thread to complete to
task defined neat in time to each other. If you have an algorithm for
this you'll be able to define the maximum numbers of Thread you'll need.
Otherwise you allocate a separate Thread per defined task.

The consequences are that probably the easiest solution to a (more or
less) complete Cron like Scheduler is an own heart beat of one minute to
check for task to launch and a pool of Threads as big as you have task
to process. As it seems Doug Leas concurrency tools could be configured
to be as resource sparing as possible (have a look at
http://gee.cs.oswego.edu/dl/classes/EDU/oswego/cs/dl/util/concurrent/PooledExecutor.html)

-- 
Giacomo Pati
Otego AG, Switzerland - http://www.otego.com Orixo, the
XML business alliance - http://www.orixo.com


Mime
View raw message