httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Axel-Stéphane SMORGRAV <Axel-Stephane.SMORG...@europe.adp.com>
Subject RE: [users@httpd] Re: High performance apache logging
Date Thu, 06 Jan 2005 11:01:13 GMT
Having the SSL session cache on DBM is a real IO hog. I can assert that it will certainly impair
performance based on load tests I did about 6 months ago.

The tests were made on a pretty lightweight configuration (Sun V210 1xUltraSPARC III+ @ 1024
MHz with 1 Ultra160 SCSI internal disk) running Solaris 8. At the beginning of the test run
we ramped up to 160 hits per second and observed a slow reduction in the number of hits per
second until we got a total collapse of performance 8 minutes into the run. The performance
degradation coincided with a marked increase in page-out. All the way through the test we
also observed a gradual increase in disk traffic.

After having reconfigured mod_ssl to use shm instead of dbm for the session cache we re-ran
the test. The number of hits per second this time rose to a sustainable 170 hits per second
while disk IO and paging rate was virtually zero.

In all tests, the SSLMutex was set to "default" which on Solaris corresponds to pthread (at
least with Apache 2.0.49).

Based on this experience I am pretty certain that your IO problems do not stem from logging
but from the SSL session cache.

Before switching to shm for SSL session cache, I would either ask for advice from others who
have done this on Linux, or test it for myself, to make sure that this will not introduce
any other undesirable effects.

Please keep me informed of your results.

-ascs

-----Message d'origine-----
De : Josh Wyatt [mailto:josh.wyatt@hcssystems.com] 
Envoyé : mercredi 5 janvier 2005 22:23
À : users@httpd.apache.org
Objet : [users@httpd] Re: High performance apache logging

Axel-Stéphane SMORGRAV wrote:
> What do you mean by system load? Is it CPU utilisation or the number of processes waiting
for the CPU including those waiting for I/O?
> 
> In any case, I would be very surprised if adding processing of Perl scripts to the equation
would improve performance...
> 
> You most probably have a disk I/O performance problem. That can be solved by investing
in faster disks (SCSI in stead of ATA, for example), or doing striping on multiple disks (RAID0).
You could even do striping across disks on different controllers.
> 
> You did not say what kind of systems you're running.
> 
> -ascs

There are a fair number of processes; I will have to get back to you on specifics.  We get
probably 1 million hits per day per server; and 90% of those are between 7am and 5pm.  Generally
about 25-30 reqs/second.

These are IBM blade servers.  They each have dual 146GB 10k RPM disks in hardware RAID1. Each
server has dual 3.2ghz xeon CPUs, and 4GB memory.  There are 4 servers.  We're running RedHat
AS 2.1, and Oracle's apache (a 1.3.x derivative).

One thing of note, they all run SSL.  The ssl_session_cache is of type "dbm", and ssl_mutex
is of type "file".  I have also considered that there could be contention on these files as
well.

I'm lead to believe we're I/O bound because of iostat output which shows average wait times
greater than 100ms, and they're all writes:
Device:  rrqm/s wrqm/s   r/s   w/s  rsec/s  wsec/s avgrq-sz avgqu-sz   await
svctm  %util
sda7       0.00  39.00  0.00 60.00    0.00  816.00    13.60     9.30  155.00
63.33  38.00


(sda7 is the partition where the logfiles, ssl_session_cache, and ssl_mutex reside)

This frequently pushes the system CPU utilization higher, raising the load average.

I feel the disks subsystem is pretty fast, but sometimes fast is not fast enough :).  I intend
to temporarily shutdown logging on two of the four machines and study the effect.

Thanks,
Josh

> 
> -----Message d'origine-----
> De : Josh Wyatt [mailto:josh.wyatt@hcssystems.com]
> Envoyé : mercredi 5 janvier 2005 16:54 À : users@httpd.apache.org 
> Objet : High performance apache logging
> 
> Hi all,
> 
> I'm interested in hearing anyone's thoughts on better performing logging mechanisms for
apache.  My environment consists of four servers, load-balanced using round-robin DNS (which
works very well).  We are currently logging using just the standard local file logging, and
it is pushing system load pretty high due to our load.  I'm leaning toward something using
a perl script, a big logserver, and pipes...?
> 
> Thanks in advance,
> Josh
> 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
> 





---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Mime
View raw message