cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antti Koivunen <anryo...@users.sourceforge.net>
Subject Re: Added Resource Monitor in Cocoon.java
Date Thu, 13 Dec 2001 11:30:52 GMT

Gerhard Froehlich wrote:

>>From: Sylvain Wallez [mailto:sylvain.wallez@anyware-tech.com]
>>
>>
>>Berin Loritsch a écrit :
>>
>>>Sylvain Wallez wrote:
>>>
>>>
> 
> <skip/>
> 
>>>Keep in mind, that I am also working on an asynchronous command structure for
>>>Avalon, something that ActiveMonitor, Pools, etc. would all be able to take
>>>advantage of.  The basic gist is this:
>>>
>>>There are a number of maintenance/management tasks that components have to
>>>take at periodic times like ActiveMonitor, DataSourcePools, Cache implementations,
>>>etc.  The problem is that having a whole thread for control management for
>>>each of these items is too heavy, as is performing the maintenance synchronously.
>>>
>>Have you looked at java.util.Timer in JDK1.3 ? AFAIK, it is meant for
>>this kind of things.
>>
> 
> AIUI it forces for every TimerTaks an own Thread (non Deamon). When you have
> many Maintainance Taks you will have large ThreadPool and that would be a
> Performance issue again.


Actually, java.util.Timer uses a priority queue of TimerTasks, ordered 
by the next execution time, and the timer thread can be run as a daemon. 
A quote from JDK 1.3.1 javadocs:

"Implementation note: This class scales to large numbers of concurrently 
scheduled tasks (thousands should present no problem). Internally, it 
uses a binary heap to represent its task queue, so the cost to schedule 
a task is O(log n), where n is the number of concurrently scheduled tasks."


> The approach here is that you have a queue, in which you put your events and
> at the end of this queue there is maybe 1-3 threads which are popping the
> events from the queue and hand them over to the corresponding EventHandler.
> 
> Therefor you can have so many tasks if you want, but the number of threads
> is small...


This is exactly the idea and one of the key concepts of SEDA.


> 
> 
>>>Part of the lessons learned from the Staged Event Driven Architecture is that
>>>many times, you only need a small number of threads (in this case, one) to
>>>handle _all_ of your management tasks.  The SEDA architecture goes so far as
>>>to apply this to all I/O as it is asynchronous in that architecture.  The
>>>scalability of the solution is insane.  For instance, a plain HTTP server
>>>built on top of the SEDA architecture was able to handle 10,000 simultaneous
>>>connections and outperform Apache httpd using IBM JDK 1.1.8 on a Linux box
>>>with 4 500 MHz processors and a Gig of RAM.  Apache only made it to about
>>>512 simultaneous connections due to process per user limitations--not to
>>>mention several requests took much longer....
>>>
>>I definitely have to read this stuff...
>>
> 
> It's quite interesting...
> 
> 
>>>I am not proposing the SEDA architecture for Cocoon (although I might examine
>>>a simple HTTP implementation that supports Servlets later).....
>>>
>>>I am proposing the migration towards an infrastructure where maintenance
>>>commands can be performed asynchronously--and checking for source file
>>>changes is only one of those commands.
>>>
>>We have to identify what are those tasks that need to be done
>>periodically and those that don't need it.
>>
>>I would say that checking configuration file changes doesn't need to be
>>done asynchronously, for several reasons :
>>- in a production environment, these files aren't likely to change
>>often, and thus periodic check is most often a waste of time,
>>- in a development environment, people that change these files
>>immediately issue a request to check the effect of the change.
>>Synchronous (possibly deferred) check is then IMO more adequate.
>>
>>Cache management, on the contrary, can be considered the other way :
>>even in a production environment, periodic asynchronous check effecively
>>has something to do, which is purge cache entries associated to changed
>>resources. This isn't absolutely needed from a functional point of view
>>but allows to free earlier some memory/disk resources that would without
>>that have been freed later by the MRU policy.
>>
> 
> Be careful, at the end you have for every special case a special solution.
> Personally I want something more generic, which fits for the most cases.


Without getting into Cocoon internals (not an expert, _yet_ ;), in the 
long run, it would be nice to have a generic "scheduled task management" 
architecture, probably based on the event handling model. In addition to 
providing a consistent API, it should really help to improve 
scalability, and make things easier to manage (self-adjusting Pool, 
Cache implementations [with fewer threads], less external I/O, etc.)

(: Anrie ;)


> 
> 
>>>I am also going to supply a PoolResource implementation in the next version
>>>of Excalibur so that we can now *gasp* get the internal information of the
>>>Pool implementations of Cocoon's resources!  How about them apples?
>>>
>>Aahh, this would be really great :)
>>
>>
>>>But you have to start somewhere....
>>>
>>>
>>>>>If you want, I remove this stuff for now, but only for a clean reboot
of this
>>>>>issue ;-).
>>>>>
>>>>>
>>>>Let's keep it for now, until we decide if this is the right way to go or
>>>>not. I will try to spend some time for show-casing the strategy I
>>>>proposed, so people can look at both.
>>>>
>>>:)
>>>
>>>Your solution works for the file issue, but the asyncronous maintenance issues
>>>are something completely different.
>>>
>>Agree. That's why we have to carefully identify those things that need
>>asynchronous management.
>>
>>BTW, I added a new components.source.DelayedLastModified and updated
>>sitemap.Handler to showcase deferred calls to Source.getLastModified().
>>Please have a look at it.
>>
>>
> 
>   Gerhard
> 
> "Three o'clock is always too late or too early for
> anything you want to do.
> (Jean-Paul Sartre)"
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
> For additional commands, email: cocoon-dev-help@xml.apache.org
> 
> 



---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org


Mime
View raw message