apr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "William A. Rowe, Jr." <wr...@covalent.net>
Subject Re: apr_buckets_file.c:file_read + XTHREAD
Date Wed, 28 Nov 2001 04:05:27 GMT
From: "Cliff Woolley" <cliffwoolley@yahoo.com>
Sent: Tuesday, November 27, 2001 9:35 PM

> Well, here's another kludge for you: the problem is that apr_bucket_read()
> for file buckets falls into an abyss of performance as soon as the magic
> APR_MMAP_LIMIT is exceeded.  That's because we give up on mmap entirely
> and read 8kb at a time frome the whole bigass file, even if all we wanted
> in the first place was the first 32 bytes or so (say for something like
> mod_mime_magic) and we're never going to touch the other 12MB.  We still
> have to pay the penalty anyway.  What if apr_bucket_read() on a file
> bucket bigger than APR_MMAP_LIMIT would turn the file bucket into an mmap
> bucket+file bucket pair, with the mmap bucket APR_MMAP_LIMIT in size, and
> the file bucket gets the rest?  I think that would tackle this problem
> nicely.  In a case like mod_mime_magic where we're only reading the
> beginning of the file, we'd actually still be able to use sendfile()
> with the first APR_MMAP_LIMIT's worth as a header.  :)  Of course, you
> might run into a case where you're os-limited as to how much you can have
> mmaped at one time per process if we buffer a large number of buckets for
> some reason, but then the mmap will fail and we'll fall back on read()
> [the code already does that].

Better yet.

If APR_MMAP_LIMIT is exceeded, break it into n APR_MMAP_LIMIT sized buckets, 
wherein only one bucket may be mapped at a time.

What I'm suggesting is that as each bucket is consumed, it is un-memaped.
The buckets are only mmaped as they are read.  This way, we get the benefits
of mmap, without the pain of page consumption.



View raw message