httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marc Slemko <ma...@znep.com>
Subject old ZD benchmark comments
Date Thu, 22 Jan 1998 03:09:46 GMT
I haven't actually looked at ZD's previous benchmarks before in
detail because I never saw real graphs.  

Now that I look again, they have better detail online.

http://www.zdnet.com/pcmag/features/webserver/_intro.htm is their 
latest test, from... erm... July 97?  Not sure of the date.  That's
the last-modified anyway.  Oh, this tested 1.1.3.

A reasonable quality PDF version of their results is at:

    ftp://ftp.zdnet.com/pcmag/1997/0527/labsdata/in1610ba.pdf

This shows a few things.

First, Apache had decent latency throughout their tests.  

The two servers they tested on Solaris had almost identical latencies
just below the 200ms mark, and they were identical right from 1 to 60 
clients.  Whenever you see 200ms constants around TCP you should be 
suspicious; that is just obvious.  This is, if I remember, the test that
led to the documentation of the slow-start "disagreement" between Solaris
and NT.  The fact that ZD did no more than say "hmm, that looks odd"
and ask Sun doesn't give me confidence.  These are the things you need
to look at, at the protocol level if necessary, to ensure that you are
not intorducing bogus factors into your tests.  Blackbox doesn't work.

For single processor static content throughput, Apache was middle
of the pack.  About the same as Netscape Enterprise and Fasttrack
on IRIX, but below a bunch of others.  It was around 5000kbytes/sec.
IIS hit the best, hitting up to ~13000kbytes.  Everything else
(eg. FastTrack and Enterprise on NT, DGUX) topped out at ~9000kbytes.

Apache was right in there growing at a linear rate with the number
of clients, like most other servers, until it hit 12 clients, then it 
hit a cap and stopped increasing.

In their NSAPI test, Enterprise on Solairs showed almost the same
rate for both NSAPI and CGI, while on all other platforms there
was a drastic difference.  If that doesn't set off red lights...

Oh yea, I like this:

    Some products, such as O'Reilly's WebSite, have higher latency
    numbers than others because of the way the Web server handles
    incoming requests. This product goes to disk to read a fresh version
    of the requested HTML page every time a request comes in. Going to
    disk every time a page is requested results in a much slower response
    time than caching the page right away (but possibly still acceptable).

Gee, Apache had pretty good latency even in your tests but you didn't
complain about it always going to disk now did you...

Apache showed almost zippo (but did have a little bit) increase adding
another processor.  IIS showed almost zippo increase.  Enterprise on NT,
OTOH, showed a good increase to bring it right up to IIS.

IIS's bouncy results that can't really be fit to a curve concern bit a bit
about IIS.  Enterprise under NT, for example, shows a very well fit
increase.


Mime
View raw message