httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From P J <pauljfli...@gmail.com>
Subject Re: [users@httpd] Help troubleshooting performance issue, after "1000 total children" Apache no longer responds to HTTP requests. Not MaxClients issue?
Date Mon, 30 Apr 2012 17:37:48 GMT
On Mon, Apr 30, 2012 at 9:13 AM, Alexandr Normuradov <normalex@gmail.com>wrote:

> cat /proc/$(pidof -s httpd)/limitsTo troubleshoot that you should have
> at least two additional outputs from
>
> netstat -pant, with connections states
> and
> service httpd fullstatus, listing current state of all the apache
> procs/threads.
>
> What applications your Apache is serving?
> PHP? is it mod_php, mod_python, mod_perl?
>
> What the vhost access log file for the most accessed vhost is showing?
> Any pattern of slow, connections  consuming attack?
> If it is, and all tasks are in the Keep Alive wait then disable Keep
> Alive and lower the general timeout to just 7 seconds.
>
> The error "connect to listener on [::]:80" error is quite unusual.
>
> ETIMEDOUT
>    Timeout while attempting connection. The server may be too busy to
> accept new connections. Note that for IP sockets the timeout may be
> very long when syncookies are enabled on the server.
>
> cat /proc/sys/fs/file-nr
>
> cat /proc/$(pidof -s httpd)/limits
>
>
> Sincerely,
> Alexandr Normalex
>

Hi Alexandr, thanks for taking a look at this with me.

The traffic pattern for this website is at certain times of the day it
receives huge spikes of traffic in very short periods of time, trying to
tune Apache to accommodate it the best we can.

cat /proc/$(pidof -s httpd)/limits

Limit                     Soft Limit           Hard Limit           Units

Max cpu time              unlimited            unlimited            seconds

Max file size             unlimited            unlimited            bytes

Max data size             unlimited            unlimited            bytes

Max stack size            10485760             unlimited            bytes

Max core file size        0                    unlimited            bytes

Max resident set          unlimited            unlimited            bytes

Max processes             55296                55296
 processes
Max open files            1024                 1024                 files

Max locked memory         32768                32768                bytes

Max address space         unlimited            unlimited            bytes

Max file locks            unlimited            unlimited            locks

Max pending signals       55296                55296                signals

Max msgqueue size         819200               819200               bytes

Max nice priority         0                    0
Max realtime priority     0                    0

cat /proc/sys/fs/file-nr
1530    0       560543

Looking at Max open files I see what is likely the problem :)
Max open files            1024

I swear I modified this to 4096! I've changed the limit to 4096 now, I'll
double check it tomorrow. Hopefully this will be the obvious fix!

I will check service httpd fullstatus  and netstat -pant tomorrow morning
when this happens again, it happens the same time every day - it is not an
attack, the customers application receives massive amounts of connections
at certain times of the day.

I've been working with Apache for 15 years and I've never seen "connect to
listener on [::]:80" error message before, I hope it's related to reaching
Max open files.

Thanks again for your help.

--
PJ

Mime
View raw message