httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Favor <da...@davidfavor.com>
Subject Re: [users@httpd] Bulkheading apache
Date Wed, 02 Apr 2014 15:55:46 GMT
David Favor wrote:
> Pratyoosh Sharma wrote:
>> We have an external facing portal that serves multiple applications 
>> via single sign on, to bypass cross domain limitations & simplifying 
>> support we proxy all the requests through apache httpd, there are 
>> about 100 independent applications that we proxy using mod_proxy
>> In case of slow responses (due to hung threads) unforeseen situations, 
>> we cause saturation of resources on apache & cause it to die.
>>
>> We want to introduce a circuit breaker/bulkhead (timeouts are not good 
>> enough) so that outages in few application do not impact the rest.
>>
>> Anyone else having the same issue? Ideas for solution?
> 
> Many times the underlying OS setup (where Apache is running)
> can be tuned to circumvent this.
> 
> The speeds listed below are fairly simple to attain. The
> following are required for this...
> 
>    1) Dedicated hardware
> 
>       Cheapest quad core + hyperthreading is sufficient for
>       even the largest of projects
> 
>    2) Following the tuning steps below
> 
>    3) Usually takes only 3-4 hours to tune for these speeds.
> 
> Start by testing bare metal speed of OS + Apache via this...
> 
>    cd some-domain-root (we'll call it foo.com)
>    touch foo.html
>    ab -k -t 10 -n 10000000 -c 5 http://foo.com/foo.html
> 
> A correctly tuned server should return huge numbers like...
> 
>    30,000-50,000+ requests/second
> 
> Next test your CMS system... as in...
> 
>    ab -k -t 10 -n 10000000 -c 5 http://foo.com/  (notice no foo.html)
> 
> A correctly tuned WordPress system should return numbers like...
> 
>    3,000-5,000 requests/second
> 
> Until you get these numbers fixed, any other attempt at tuning will fail 
> miserably.
> 
> To get these huge WordPress speeds requires tuning ever subsystem, bare 
> metal upwards...
> 
>    1) Raw disk devices (filesystem + mount options)
> 
>    2) /tmp must be put in tmpfs (/var/tmp must stay on disk)
> 
>    3) Use MariaDB (MySQL that works + works fast) + tune it well
> 
>    4) Tune the Apache MPM you're using...
> 
>       Till you get the 30,000-50,000+ requests/second static file speed.
> 
>    5) Tune PHP Opcache till you see 99% hits out of cache, with
>       lots of free memory left over.
> 
>       Keep in mind, since PHP-5.5 APC is no longer used, Opcache
>       replaces APC. This bites many people as they update Apache/PHP.
> 
>    6) Tune WordPress theme + plugins + caching
> 
>       Till you get the 3,000-5,000 requests/second 
> PHP/CMS/WordPress/Whatever speed.
> 
>    7) Track Opcache usage + increase memory as required.
> 
>    8) Track top/swapping + add more RAM as required.
> 
> In most cases, it's far cheaper + easier to manage large sites by...
> 
>    1) Doing the tuning above
> 
>    2) Changing Apache ServerLimit till all Apache crashes + weirdness 
> disappears
> 
>    3) Adding memory to Opcache + System RAM produces far better results 
> than
>       complex frontends.
> 
> The big winner is making sure Opcache has high hit rate + extra memory...
> 
> And there is no swapping.

One other thing to avoid. Apache suexec appears to degrade performance in some
cases. Haven't tracked this down yet + the easy work around is this...

1) set mount options on disk to add 'acl'

2) set all Apache directories to default Apache user:group ownership + permissions

3) use setfacl to add handle sandboxing + allowing access to different
    Apache installation data files, like with ftp or sftp.

Using extended ACLs seems to produce no measurable performance reduction + it's
way easier to manage than suexec setups.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Mime
View raw message