httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From P J <pauljfli...@gmail.com>
Subject [users@httpd] Help troubleshooting performance issue, after "1000 total children" Apache no longer responds to HTTP requests. Not MaxClients issue?
Date Mon, 30 Apr 2012 15:20:13 GMT
Greetings all,

Hoping someone can point me in the right direction as I've spent the last
week trying to figure out where the "issue" is but haven't been able to.

Running Apache 2.2.3 on CentOS 5.8.

At a few points during the day when traffic is heavy we are having an issue
where Apache no longer responds to any HTTP requests.

It sounded like a standard MaxClients being reached issue, but
that doesn't seem to be the case.

Also,, logging into the machine during this time the load average is under
1, and there is still plenty of RAM available.

Reviewing /var/log/httpd/error_log I've noticed the following pattern:

--snip--
[Mon Apr 30 07:00:34 2012] [info] server seems busy, (you may need to
increase StartServers, or Min/MaxSpareServers), spawning 32 children, there
are 0 idle, and 905 total children
[Mon Apr 30 07:00:35 2012] [info] server seems busy, (you may need to
increase StartServers, or Min/MaxSpareServers), spawning 32 children, there
are 0 idle, and 937 total children
[Mon Apr 30 07:00:36 2012] [info] server seems busy, (you may need to
increase StartServers, or Min/MaxSpareServers), spawning 32 children, there
are 0 idle, and 969 total children
[Mon Apr 30 07:00:37 2012] [info] server seems busy, (you may need to
increase StartServers, or Min/MaxSpareServers), spawning 32 children, there
are 35 idle, and 1001 total children
[Mon Apr 30 07:00:42 2012] [debug] mpm_common.c(663): (70007)The timeout
specified has expired: connect to listener on [::]:80
[Mon Apr 30 07:00:49 2012] [debug] mpm_common.c(663): (70007)The timeout
specified has expired: connect to listener on [::]:80
[Mon Apr 30 07:00:56 2012] [debug] mpm_common.c(663): (70007)The timeout
specified has expired: connect to listener on [::]:80
[Mon Apr 30 07:01:03 2012] [debug] mpm_common.c(663): (70007)The timeout
specified has expired: connect to listener on [::]:80

A few times a day, right after "1000 total children" Apache stops
responding and has to be restarted in order to work again.

I've reviewed the error_log from a few weeks back and it's the same
pattern, the server hits 1000 total children and then immediately spits out
the
"[debug] mpm_common.c(663): (70007)The timeout specified has expired:
connect to listener on [::]:80" error message and stops responding.

Yet the load on the server is quite low...

Here is the relevant section from the config:

Timeout 45
KeepAlive On
MaxKeepAliveRequests 10000
KeepAliveTimeout 3

<IfModule prefork.c>
StartServers      80
MinSpareServers   50
MaxSpareServers  120
ServerLimit     3500
MaxClients      3500
MaxRequestsPerChild  2000
</IfModule>


Anyone know why the magic number of children Apache is able to reach is
1000 before it stops processing more requests?
Or how to make sense of the "(70007)The timeout specified has expired:
connect to listener on [::]:80" message?

What "timeout specified" is it referring to? The 45 seconds?

Any help would be greatly appreciated.

Thanks in advance.
PJ

Mime
View raw message