httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Stoddard" <b...@wstoddard.com>
Subject RE: mod_mem_cache bad for large/busy files (Was: [PATCH] remove some mutex locks in the worker MPM)
Date Thu, 02 Jan 2003 15:06:56 GMT
Random thoughts:
- Did the content have short expiration times (or recent change dates which
would result in the cache making agressive expiration estimates). That could
churn the cache.
- Was CacheMaxStreamingBuffer set appopropriately? (it may not be needed at all
if the content length header is included on all replies).
- Did you try caching open file descriptors? I am rather curious if caching open
fds will be useful/practicle on Unix systems.  Oh..., but this probably will not
help your disk throughput... nevermind.
- It's probably worth noting in the doc that -each- child process will cache up
to MCacheSize KBytes.  If you have 10 child processes, then you need
10xMCacheSize Kbytes RAM available just for the cache (the same files could be
cached in each process). I wonder if we should, at startup, allocate MCacheSize
KB of shared storage and have mod_mem_cache allocate out of the shared pool.
Each child process would have it's own unique reference to the object, but the
object itself would only be cached once for all processes to access.

Bill

> Apache 2.0.43, Solaris 8, Sun E220R, 4 gig memory, gig ethernet.  We tried
> both Sun forte and gcc compilers.  The problem was mod_mem_cache was just
> way too resource intensive when pounding on a machine that hard, trying to
> see if everything would fit into the cache... cpu/mutexes were very high,
> especially memory was out of control (we had many very large files, ranging
> from half dozen to two dozen megs, the most popular of those were what we
> really wanted cached), and we were running several hundred concurrent
> connections at once.  Maybe a new cache loading/hit/removal algorithm that
> works better for many hits to very large files would solve it I dunno.
>
> We finally settled on listing out some of the most popular files out in the
> httpd.conf file for mod_file_cache, but that presents a management problem
> as what's most popular changes.  It would have been nicer if apache could
> auto-sense the most popular files.  Also it seems mod_file_cache has a file
> size limit but at least we got enough in there the disk wasn't bottlenecking
> anymore...
>
> Dave
>
> ----- Original Message -----
> From: "Bill Stoddard" <bill@wstoddard.com>
> To: <dev@httpd.apache.org>
> Sent: Wednesday, January 01, 2003 6:38 AM
> Subject: RE: [PATCH] remove some mutex locks in the worker MPM
>
>
> > > it may also have to do with caching we were doing (mod_mem_cache crashed
> and
> > burned,
> > What version were you running?  What was the failure? If you can give me
> enough
> > info to debug the problem, I'll work on it.
> >
> > Bill
> >


Mime
View raw message