httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeroen Geilman <jer...@adaptr.nl>
Subject Re: [users@httpd] Apache 2.2 connections full
Date Tue, 07 Jun 2011 22:25:58 GMT
On 06/07/2011 10:59 PM, Rob Morin wrote:
>
> Hello all was not sure how to word my subject line...
>
> I posted a few weeks back on how could I increase performance of 
> apache... I received quite a few replies and they all helped a bit...
>
> We even added an additional server to our server cluster to total 7 
> web servers now, all load balanced by HAProxy
>
> We currently receive about 14 million impressions a day to about 60 
> different domains/websites....
>
> Our hosting provider uses HAProxy, rather than an appliance,  I assume 
> to reduce costs. However I feel like these HAProxy machines are not 
> reliable at all....Well not under the supervision of our hosting 
> company , anyways...J
>
> On to my questions....
>
> During peek hours, like a few mins ago, all of our web servers became 
> high in load, like close to 100! We still had plenty of RAM left and 
> for sure there was no swapping....
>

So what was the cause of the load ?

Examine dstat output while the load spikes and see if there is a high 
percentage spent in IOWait.

Apache does ultimately serve content from disk; is your disk subsystem 
up to the task ?

> Please see my config below as  quick snapshot of the httpd.conf file 
> on our Centos Servers.
>
> The weird thing is when I would go see the server-status I would see 
> all the slots filed with either W or C and nothing else.... Then after 
> a what appears a random amount of time it would all clear up , and 
> then come back again. Almost like HAProxy would send a bunch of 
> requests to a server, then it would get full and then(haproxy) say, oh 
> that server is filled I will not send traffic there, for now, then it 
> would come back and see hey this server is  empty, lets send a bunch 
> of traffic there...
>

So it may just as well be the frontend that causes these spikes; you 
want to establish this with certainty.

Do all apache backend boxes spike at the same time ?

You really need to do a more thorough investigation.

Consider graphing the performance of all boxes, and comparing values 
during different times of the day.

An rrdtool grapher like cacti or mrtg will help with this, at marginal 
extra load.

If you consider that too heavy, munin has useful data to correlate CPU 
with disk I/O and apache stats.



> We also use memcache to cache sessions and mysql queries, along with 
> eAcclerator... I assume that those are functioning properly
>


That's a pretty large assumption. Don't.

> Does this make sense, am I on drugs? J
>

You tell us.

> <IfModule prefork.c>
> <IfModule worker.c>

This tells us nothing. WHICH ONE are you actually using ?
If you're using prefork (which is the stupid choice for high load 
machines), starting up a hundred new threads at peak times will take 
MINUTES.

It looks as if you haven't tuned any of the really vital server 
variables for this supposed high load you're trying to handle.


> <Directory />
>
>     Options FollowSymLinks
>
>     AllowOverride None
>
> </Directory>
>

You really want to deny access here.

> <Directory "/home/www">
>
>     Options -Indexes FollowSymLinks
>

This does nothing.
>
>     AllowOverride All
>
>     Order allow,deny
>
>     Allow from all
>
> </Directory>
>
> <IfModule mod_userdir.c>
>
>     UserDir disable
>
> </IfModule>
>
> DirectoryIndex index.php index.html
>
> AccessFileName .htaccess
>
> <Files ~ "^\.ht">
>
>     Order allow,deny
>
>     Deny from all
>
> </Files>
>
> TypesConfig /etc/mime.types
>
> DefaultType text/plain
>
> <IfModule mod_mime_magic.c>
>
>     MIMEMagicFile conf/magic
>
> </IfModule>
>
> HostnameLookups Off
>
> ErrorLog logs/error_log
>
> LogLevel notice
>
> LogFormat "%h" combined
>
> LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b" common
>
> LogFormat "%{Referer}i -> %U" referer
>
> LogFormat "%{User-agent}i" agent
>
> CustomLog "/dev/null" combined
>

That's asenine.

> GeoIPEnable On
>
> GeoIPDBFile "/usr/local/share/GeoIP/GeoIPCity.dat"
>
> GeoIPScanProxyHeaders On
>

Is this in any way a heavy/expensive database ?
What is being logged by this ?
What is it used for ?
Depending on usage, it may be as bad as performing DNS lookups.


-- 
J.


Mime
View raw message