cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bruno Dumon <br...@outerthought.org>
Subject Re: CommandManager issues [was Re: Releasing 2.1.1?]
Date Tue, 02 Sep 2003 16:19:00 GMT
On Tue, 2003-09-02 at 15:52, Berin Loritsch wrote:
> Bruno Dumon wrote:
> 
> 
> > 
> > Or maybe not so great. I'm not sure the CommandManager is well suited
> > for a general purpose scheduler.
> > 
> > What follows now should be read with a 'AFAIU' disclaimer:
> > 
> > All commands added to the CommandManager are executed sequentially and
> > on one thread. So if there is one command that takes somewhat longer to
> > execute, it will block the execution of other commands. This may be fine
> > for management-tasks which should be executed repeatedly (but not at
> > fixed times), but for a scheduler I would expect a command to be
> > executed at the configured time.
> 
> The number of threads is configurable.  By default it is based on a function
> of the number of processors.  In Cocoon it is currently 1 thread per processor
> (plus one to manage the queue).  If you have two processors, you have two
> threads.  Alternatively you can implement two threads per processor (works
> well in most situations).  This approach makes sure that long running tasks
> won't block all other items in the queue.
> 
> The Scheduler/ThreadPool combo you have with the Cornerstone components does
> not give you anything more in this area.  The queue is moved to the Scheduler,
> and the ThreadPool will block until a thread is made available.  The main
> difference is that most folks typically assign more threads to the ThreadPool
> than you have assigned to the ThreadManager.
> 
> > This could be solved by creating a different CommandManager for each
> > scheduled task, since multiple CommandManagers can be executed in
> > parallel. But then you'd still want to use a different ThreadManager
> > than the TPCThreadManager, since that will block when all threads are in
> > use.
> 
> Huh?  No.  The ThreadManager does the actual execution.  The CommandManager
> merely organizes the commands and sets them up in the queue.
> 
> 
> > The result will however be rather overkill IMHO: even commands that need
> > to be executed only weekly, will be checked every 100 ms (= sleeptime
> > parameter) to see if they need to be executed.
> > 
> > Conclusion: while this event-based infrastructure is quite cool, I'm not
> > convinced yet it is good for scheduling-purposes.
> 
> No matter how you slice it, even the Scheduler has this tradeoff.  You can
> set the sleeptime longer if you never have anything checked less then 5 second
> intervals or minute intervals, but in the end, the overhead of checking the
> commands every 100 ms is so minimal that you would never see it.  Esp. since
> that checking is done asynchronously from your critical path.  There is a
> dedicated thread in each case that makes the checks for you--essentially
> the only difference is in how many other components are needed to get the
> job done.

Berin,

I don't want to spend very much time on this thread anymore, but here's
the short summary:

I don't have any problem with the concepts behind the
CommandManager/ThreadManager or the event package in general, in fact I
think they're very good. My last point about the continuous checking of
events every 100 ms isn't really a big issue for me.

I only wanted to point out some problems in the current implementation
as it is. Those problems can of course be fixed, and for Cocoon to
depend on this system, they also need to be fixed. And maybe I'm also
wrong on some of these problems.

One of these problems is the above mentioned fact that all events from
one CommandManager are executed on one thread, no matter how many
threads you configure in the ThreadManager. I got this only from looking
at the code, here's the explanation I mentioned earlier in this same
thread, I hope you can prove me wrong:

If you look at the code of the AbstractThreadManager:

In the method "register( EventPipeline pipeline )" one PipelineRunner is
created to handle events from the given EventPipeline (e.g. the one of
the CommandManager).

Then in the run() method the PipelineRunners are executed in parallel by
using the PooledExecutor: m_executor.execute( nextRunner );

But in the PipelineRunner.run() method, and the
CommandManager.handleEvents() which is called by it, no new threads are
started.

-- 
Bruno Dumon                             http://outerthought.org/
Outerthought - Open Source, Java & XML Competence Support Center
bruno@outerthought.org                          bruno@apache.org


Mime
View raw message