httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "William A. Rowe, Jr." <>
Subject Re: cvs commit: httpd-2.0/modules/http mod_mime.c
Date Wed, 15 Aug 2001 22:34:14 GMT
From: "Greg Ames" <>
Sent: Wednesday, August 15, 2001 4:21 PM

> wrote:
> > 
> > wrowe       01/08/14 20:09:33
> > 
> >   Modified:    modules/http mod_mime.c
> >   Log:
> >     Solve the segfault until the right patch is unearthed.
> >     [Greg Ames]

[the right patch is attached]

> slower doing directory merges during config file parsing perhaps (who
> cares?), but faster while serving most requests due Brian's hash stuff.  

I care, since some mechansims (think subrequests) -may- generate hundreds of
merges.  The nature of rebuilding hashes also has exponential side effects.

That wasn't the only remaining segfault.  Mucking with a specific exinfo entry
within a subreq config would STILL cause segfaults in the request or a later subreq.
(Think in terms of includes, adding an incremental .shtml on a first include'ed file, 
so when the second include'ed file is loaded the segfault is triggered.)

It has to be done right, or return to tables :(

Would you please apply the attached patch on Daladeus and let me know if the 
segfaults are gone, while I work up documentation of the rules of when/how one 
can optimize their directory-merge?  Without an agreed framework, this will never

> The config can be dynamically updated during request/subrequest
> processing due to .htaccess files or <File *.asc> containers, maybe a
> few other cases.  You want to avoid .htaccess anyway for fastest
> performance (see if you're
> not convinced).  Plus, it seems like whenever the config is being
> updated dynamically, there won't be a whole lot of cases where we don't
> need a new copy of the hash table.

Huh?  I'll post a seperate document describing the gains.

> Brian's suggestion to cache the results of directory merges sounds like
> a more general solution to the problem, assuming there really is a
> problem.  

Yes, yes, yes :)  But that doesn't prevent us from doing the 'right thing' on both counts.
Let's say we set up a max of 100 most-frequent directories for caching.  On a huge webserver,
that leaves many directory walks incomplete (or never started.)

> Have directory merges actually shown up on the radar screen
> while doing profiling?

Of course, that was why Brian offered up the original (unfortunately broken) patch.
Optimization is confusing, that's why I'm going to write a short, clear description
of the do's and dont's so we and module authors start from the same assumptions.

> p.s. if somebody wants to make autoindexes run a lot faster (not sure I
> care), why don't we cache which icon to present for a given file?  I
> mention autoindexes because that's what triggered this bug.

Because it's already there, and if this patch cleans up the behavioral problems, it should
be extremely fast.  Icons are allowed to vary by <Directory>, <File>, and <Location>

> On daedalus, that ends up driving mod_mime_magic for many files every
> single time a browser accesses a distribution directory.  This generates
> tons of overhead, including gunzip'ing at least part of every .gz file
> in the directory.  I suppose we should try to shake out bugs in m_m_m,
> but we can do that without constantly driving it to choose the autoindex
> icons.

Caching won't fix that.  We should look at which requests fall back on mod_mime_magic, it
really shouldn't be hit so often :(  I'll work up some info-level logging patch for m_m_m
it's such a performance killer.  Autoindex works up a subrequest for many reasons, even checking
that the file can be served.  If you kill the subrequests from autoindex, we will loose the
appearance of security, even if the actual files can't be comprimized (easily.)


View raw message