cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sylvain Wallez <sylvain.wal...@anyware-tech.com>
Subject Re: Adding Resource Monitor to Generators
Date Mon, 10 Dec 2001 16:41:18 GMT


Berin Loritsch a écrit :
> 
> Sylvain Wallez wrote:
> 
<snip/>
> >
> > But here's another use case : suppose you have a portal site where 10%
> > of the pages make 90% of the requests, but where 100% of the pages are
> > monitored, because they have been visited at least once since server
> > startup. I'm not good at statistics, but lets suppose this results in
> > only 15% of all pages to be requested in a monitor scan period.
> >
> > This means that at each refresh period, ActiveMonitor will scan 100% of
> > the pages when only 15% are really needed to be scanned : 85% of useless
> > File.getLastModified() !! This is what I wanted to point out.
> 
> My concept of how to use Monitor was to monitor things that require
> action.  This means that XSP and Sitemap files should be monitored--their
> change requires action by the system.  Cached resources must be monitored,
> as they affect the validity of the cache.  The action I foresee for this
> event is to invalidate the cache entry and release the Resource.
> 
> This way only the pages that are required to be monitored are monitored.
> 
> As to the 85% useless File.getLastModified(), you have to consider the
> cost of the action.  If you have that many resources that you have not
> removed your Resource entry, then there is something wrong in the way
> you are using it.  Also, if this is happening in a background thread
> at low priority (read asynchronously that will not override actual
> serving of requests), then the cost of these extraneous calls are lowered!

OK I think to have understood now (light bulb over my head;) : do you
mean that once a Resource has been detected as modified, we should
remove it from the monitor and release the associated data, i.e. remove
the cache entry or unload the XSP-generated class ? This makes sense,
now.

One more question : don't you think a kind of Resource time-to-live can
be useful, i.e. automatically invalidate resources that haven't been
used for a long time ? This would allow to purge infrequently used
resources from the system. Or maybe this should be handled by the MRU
policy of the cache.

> Do you need to monitor resources in a ClassLoader?  most likely not.  You
> merely have to assume that they do not change--which for a production
> system is most likely true.
>
> >>>It seems to me that the main benefit of ActiveMonitor is for resources
> >>>that are systematically checked at each and every request : IMO, this
> >>>should be limited to configuration files and sitemaps.
> >>>
> >>Yes and also systematically checked are the Generators. On every request
> >>the CachingStreamPipeline validates TimeStampCacheValidity. TimeStampCacheValidity
> >>is set i.e. from the FileGenerator, which calls getLastModified() every
> >>request. TimeStampCacheValidity signals if a Source (i.e sample.xml)
> >>has changed or not.
> >>
> >>
> >>>For less-frequently used resources, wouldn't it be a better solution to
> >>>only call getLastModified() when the resource is actually used and the
> >>>time since the last call to getLastModified() is greater than the
> >>>refresh period ? This would be a kind of buffering in front of the
> >>>filesystem. Also, can't this be integrated directly in Source ?
> >>>
> >>Hmm I don't get you here ;).
> >>
> >
> > Ok, maybe some code will be more clear ;)
> >
<snip-code/>
> >
> > This ensures the LastModified information isn't older that the refresh
> > period, and that - and this what I wanted to explain - refresh occurs
> > only when the information is actually requested.
> >
> > If a FileResource is used once a week, File.lastModified() will be
> > called only once a week even if refreshPeriod is 1 hour, while in the
> > same conditions ActiveMonitor will call it 24*7 = 168 times !
> >
> > Note also that the above algorithm can really easily be integrated into
> > AbstractSource.
> >
> > Thoughts ?
> 
> ActiveMonitor is a tool for developing apps, you might not want it all
> the time.  Also, ActiveMonitor is best used in systems where action
> must be taken on the resource.  I would rather the ProgramGenerator
> and Cache system uses it, and all other Generators and Transformers,
> etc. never make _any_ calls to getLastModified().  That way, if the
> cache entry expires, the Resource reference is released, and no other
> Generator/Transformer/etc. needs to be notified.

Cool. My fears and wonders are vanishing :)

Sylvain.

> >>>Last point : your changes in ProgramGenerator make the assumption that
> >>>sources are files. This won't be true in unexpanded war files and will
> >>>very likely break the engine ;)
> >>>
> >>But how works the FileGenerator and his related, when they call
> >>getLastModified() from the Source in this case?
> >>
> >
> > Using URLConnection.getLastModified(), whose abilities highly depend on
> > the protocol handler.
> >
> >
> >>Cheers
> >>Gerhard
> >>
> >
> > Sylvain.

-- 
Sylvain Wallez
Anyware Technologies - http://www.anyware-tech.com

---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org


Mime
View raw message