httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Orton <jor...@redhat.com>
Subject Re: Possible new cache architecture
Date Wed, 03 May 2006 11:46:53 GMT
On Tue, May 02, 2006 at 02:21:27PM +0200, Plüm, Rüdiger, VF EITO wrote:
> Another thing: I guess on systems with no mmap support the current implementation
> of mod_disk_cache will eat up a lot of memory if you cache a large local file,
> because it transforms the file bucket(s) into heap buckets in this case.
> Even if mmap is present I think that mod_disk_cache causes the file buckets
> to be transformed into many mmap buckets if the file is large. Thus we do not
> use sendfile in the case we cache the file.
> I the case that a brigade only contains file_buckets it might be possible to
> "copy" this brigade, sent it up the chain and process the copy of the brigade
> for disk storage afterwards. Of course this opens a race if the file gets
> changed in between these operations.
> This approach does not work with socket or pipe buckets for obvious reasons.
> Even heap buckets seem to be a somewhat critical idea because of the added memory usage.

The way I would expect it to work would be by passing f->next in to the 
store_body callback, it looks doomed to eat RAM as currently designed. 
mod_disk_cache's store_body implementation can then do:

 1. read bucket(s) from brigade, appending to some temp brigade
 2. write bucket(s) in temp brigade to cache file
 3. pass temp brigade on to f->next
 4. clear temp brigade to ensure memory is released
 5. goto 1

joe

Mime
View raw message