httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Evans <tevans...@googlemail.com>
Subject Re: [users@httpd] apache hanging whole system
Date Mon, 17 May 2010 17:07:14 GMT
On Sun, May 16, 2010 at 9:42 PM, Peter Halicky <peto@halicky.sk> wrote:
> Hi,
>
> I run a "LAMP" server with a few websites, one of them quite busy. I use
> Ubuntu 8.04 with apache 2.2.8
> and PHP 5.2.4. The problem is this: about once a day, apache processes
> cause the system load to go very
> high (today it was ~30). During this time, I can not display any webpage
> on my server. If I do not notice, the
> problem goes away after 3-10 minutes. If I do notice, I can ssh into the
> server (if I'm lucky and I get a connection,
> which could be a problem with sysload over 30+), I kill all apache
> processes (very many of them, usually equal
> to MaxClients), restart and everything is fine (with sysload rarely
> above 2, usually around 0.5).
>
> I use the prefork MPM with the following settings (just for the record):
>
>    StartServers          5
>    MinSpareServers       5
>    MaxSpareServers      10
>    MaxClients           80
>    MaxRequestsPerChild  10
>
> I have also reduced the KeepAliveTimeout to 3 seconds.
>
> The logs did not show anything interesting or out of the ordinary. Also
> this happens quite randomly, not really
> in the busiest hours (yesterday it was at 6am, which is the least busy
> time for the server).
>
> I configured a mod_status URL which I am monitoring, to see what is
> making apache so busy. Unfortunately,
> when this happens, apache is so busy that it doesn't respond to these
> requests...
>
> Any advice would be much appreciated.
>
> Thanks!
> Peter
>

Apache is unlikely to hang your whole machine - its normally the 'MP'
part that has the problems. I don't know about Linux issues though.

P:

PHP uses a lot of memory. Run your server for a while, and look at the
RSS/RES size of the httpd processes. A 'stock' apache process should
be around 5-10Mb. A process with a nice PHP interpreter can be
anything from 10Mb to 200Mb - check to see how much yours are.

When your server gets busy, you're telling apache that it can spawn 80
children. Does 80 * <avg process size> fit in your RAM?

M:

mysql has locks. It is possible that your web application has locked
up the DB server in some way - a long running update, or a DB dump,
etc. If each request coming into the webserver requires a query that
is blocked, then apache will have to start spawning more children, as
the requests aren't being completed appropriately.
This will very quickly turn into a 'perfect storm' :

queries are blocked in mysql -> children dont finish requests ->
apache spawns more children -> more queries submitted to mysql ->
slower operating mysql -> more queries are blocked in mysql

This can quickly lead to resource exhaustion. You wouldn't be able to
connect to apache (no spare children) and there is so little RAM
available that login cant spawn a tty. You may be just about able to
ping the box.


A solution is to not serve PHP in this manner. PHP as FastCGI works
well, and clearly indicates where the memory is going (eg to PHP or
Apache) and you can put tighter controls on FastCGI (which would lead
to slower response times to your webapp, rather than overloading your
server) than you would want to on apache.

You could even then run apache with a more resource efficient and
effective MPM, like worker or (my favourite) event. Apache with the
event MPM serve all of $JOBs web apps static files and reverse proxies
to all our app servers, and load never goes above 0.1, ram usage of
apache never above 150Mb, serving between 2 and 4 million requests a
day.

Cheers

Tom

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Mime
View raw message