httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Morrissey <...@horde.net>
Subject Re: [users@httpd] Excessive memory consumption
Date Fri, 14 Jul 2006 12:09:14 GMT
Hi Joshua--

On Tue, Jun 06, 2006 at 11:12:17AM -0400, Joshua Slive wrote:
> On 6/6/06, John Morrissey <jwm@horde.net> wrote:
> > I'm having a problem with Apache 2.0.54 (we're running the Debian
> > packages for 2.0.54-5). Periodically, a single Apache child will rapidly
> > consume all available memory on the machine - once the memory
> > consumption starts, it only takes five or ten minutes (at most) for it
> > to exhaust available memory.
> 
> Change your LogFormat to include the PID of the serving process.  Then
> you can go back and correlate teh PID of the memory hog with the
> requests it recently served.  (See also mod_log_forensic, which will
> be necessary if the bad request never finishes.)

Thanks for your help. I've tried both, and it seems the errant children are
only serving static content and mod_rewrite-generated redirects (returning
302s). The forensic logs back this up; there are only a handful of
uncompleted requests each day, all for static content.

Interestingly, it seems some children continue to serve requests while
leaking huge amounts of RAM. Here's a recent example. Our errant process
killer activated at 23:36:24. sar(1) says:

time     kbramused %swpused
22:45:04 645516    0.48
22:55:05 2010532   0.48
23:05:04 2025148   50.44
23:15:02 2025696   55.02
23:36:26 769572    3.58

So sometime between 22:45 and 22:55, this child began leaking. The access
logs indicate this child continued to serve static content (except for one
exception, see below) during this period, ~375 requests more or less evenly
spaced over time.

The only exception to this even spacing is thus: at 23:25:41, it served a
redirect, then at 23:36:20 apparently served its last request (484 bytes of
static HTML) before being killed. I suspect this gap is because the machine
became so RAM-tight that it was swapping heavily.

The only dynamic content served by this child during this time period is a
call to wwwcount (http://www.muquit.com/muquit/software/Count/Count.html) at
22:46:46, but I'm inclined to discount this as coincidence since the same
child served identical wwwcount requests at 22:25:35, 22:23:52, 22:03:40,
22:01:21 (... ad infinitum). Also, wwwcount requests account for a
nontrivial amount of these machines' traffic, but we're only seeing a couple
errant processes a day, at most.

I'm at a loss. FWIW, here's a link to my original post, since it's been
about a month:

http://marc.theaimsgroup.com/?l=apache-httpd-users&m=114960657316006&w=2

Any other ideas, or things I can try?

thanks,
john
-- 
John Morrissey          _o            /\         ----  __o
jwm@horde.net        _-< \_          /  \       ----  <  \,
www.horde.net/    __(_)/_(_)________/    \_______(_) /_(_)__

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Mime
View raw message