apr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Bloom <...@covalent.net>
Subject Re: apr_buckets_file.c:file_read + XTHREAD
Date Wed, 28 Nov 2001 05:26:46 GMT
On Tuesday 27 November 2001 08:05 pm, William A. Rowe, Jr. wrote:
> From: "Cliff Woolley" <cliffwoolley@yahoo.com>
> Sent: Tuesday, November 27, 2001 9:35 PM
>
> > Well, here's another kludge for you: the problem is that
> > apr_bucket_read() for file buckets falls into an abyss of performance as
> > soon as the magic APR_MMAP_LIMIT is exceeded.  That's because we give up
> > on mmap entirely and read 8kb at a time frome the whole bigass file, even
> > if all we wanted in the first place was the first 32 bytes or so (say for
> > something like mod_mime_magic) and we're never going to touch the other
> > 12MB.  We still have to pay the penalty anyway.  What if
> > apr_bucket_read() on a file bucket bigger than APR_MMAP_LIMIT would turn
> > the file bucket into an mmap bucket+file bucket pair, with the mmap
> > bucket APR_MMAP_LIMIT in size, and the file bucket gets the rest?  I
> > think that would tackle this problem nicely.  In a case like
> > mod_mime_magic where we're only reading the beginning of the file, we'd
> > actually still be able to use sendfile() with the first APR_MMAP_LIMIT's
> > worth as a header.  :)  Of course, you might run into a case where you're
> > os-limited as to how much you can have mmaped at one time per process if
> > we buffer a large number of buckets for some reason, but then the mmap
> > will fail and we'll fall back on read() [the code already does that].
>
> Better yet.
>
> If APR_MMAP_LIMIT is exceeded, break it into n APR_MMAP_LIMIT sized
> buckets, wherein only one bucket may be mapped at a time.
>
> What I'm suggesting is that as each bucket is consumed, it is un-memaped.
> The buckets are only mmaped as they are read.  This way, we get the
> benefits of mmap, without the pain of page consumption.
>
> Idea?

Not good.  The problem is that you will take the real chance that multiple
filters will be converting to mmap, and then unmapping large files without
making any changes.  Think of a large SSI file that does have any tags.
You'll MMAP each piece, unmapping as you go, and then you'll do the
same thing again in the next filter.

Ryan

______________________________________________________________
Ryan Bloom				rbb@apache.org
Covalent Technologies			rbb@covalent.net
--------------------------------------------------------------

Mime
View raw message