httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Davi Arnaut <>
Subject Re: svn commit: r467655 - in /httpd/httpd/trunk: CHANGES docs/manual/mod/mod_cache.xml modules/cache/mod_cache.c modules/cache/mod_cache.h
Date Wed, 25 Oct 2006 22:00:04 GMT
Davi Arnaut wrote:
> Graham Leggett wrote:
>> Davi Arnaut wrote:
>>> Yes, first because size_t is 32 bits :). When you do a read like this on
>>> a file bucket, the whole bucket is not read into memory. The read
>>> function splits the bucket and changes the current bucket to refer to
>>> data that was read.
>> 32 bits is 4GB. A large number of webservers don't have memory that 
>> size, thus the problem.
>>> The problem lies that those new buckets keep accumulating in the
>>> brigade! See my patch again.
>> Where?
> I was referring to "it will attempt to read all 4.7GB". Such a thing
> does not exist!
>> We start with one 4.7GB bucket in a brigade.
> No, we start with a file bucket on the brigade, I will try to explain
> what happens to see if we are talking about the same thing.
> Suppose we have a brigade containing a a file bucket, and the file size
> is 4.7GB. We want to read it fully.
> When we call apr_bucket_read() on this bucket, we end-up calling the
> bucket read function (file_bucket_read). What does the bucket file read do ?
> If mmap is supported, it mmaps APR_MMAP_LIMIT bytes (4MB) by creating a
> new mmap bucket and splits the bucket. So, after calling read on a file
> bucket you have two buckets on the brigade. The first one is the mmap
> bucket and last is file bucket with a updated offset.

s/last/last one is the old/

> The same thing happens if mmap is not supported, but the bucket type
> will be a heap bucket. If we don't delete or flush those implicitly
> created buckets they will keep the whole file in memory, but one single
> read will not put the entire file on memory.

s/keep the whole file in memory/will accumulate on the brigade, implying
that the whole file is in memory/

> What Joe's patch does is remove this first implicitly created bucket
> from the brigade, placing it on the brigade on a temporary brigade for
> sending it to the client.

s/on the brigade//

> That's why splitting the brigade with magical values (16MB) is not such
> a good idea, because the bucket type knows betters and will split the
> bucket anyway.

Davi Arnaut

View raw message