httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Graham Leggett <>
Subject mod_cache: store_body() bites off more than it can chew
Date Thu, 02 Sep 2010 17:16:43 GMT
Hi all,

An issue with mod_cache I would like to address this weekend is the  
definition of the store_body() function in the cache implementation  

     apr_status_t (*store_body)(cache_handle_t *h, request_rec *r,  
apr_bucket_brigade *b);

Right now, mod_cache expects a cache implementation to swallow the  
entire bucket brigade b before returning to mod_cache.

This is fine until the bucket brigade b contains something really  
large, such as a single file bucket pointing at a 4GB DVD image (such  
a scenario occurs when files on a slow disk are cached on a fast SSD  
disk). At this point, mod_cache expects the cache implementation to  
swallow the entire brigade in one go, and this can take a significant  
amount of time, certainly enough time for the client to get bored and  
time out should the file be large and the original disk slow.

What I propose is a change to the function that looks like this:

     apr_status_t (*store_body)(cache_handle_t *h, request_rec *r,  
apr_bucket_brigade *in, apr_bucket_brigade *out);

Instead of one brigade b being passed in, we pass two brigades in, one  
labelled "in", the other labelled "out".

The brigade previously marked "b" becomes "in", and the cache  
implementation is free to consume as much of the "in" brigade as it  
sees fit, and as the "in" brigade is consumed, the consumed buckets  
are moved to the "out" brigade.

If store_body() returns with an empty "in" brigade, mod_cache writes  
the "out" brigade to the output filter stack and we are done as is the  
case now.

Should however the cache implementation want to take a breath, it  
returns to mod_cache with unconsumed bucket(s) still remaining in the  
"in" brigade. mod_cache in turn sends the already-processed buckets in  
the "out" brigade down the filter stack to the client, and then loops  
round, calling the store_body() function again until the "in" brigade  
is empty.

In this way, the cache implementation has the option to swallow data  
in as many smaller chunks as it sees fit, and in turn the client gets  
fed data often enough to not get bored and time out if the file is  
very large.


View raw message