httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From di...@covalent.net
Subject Can AB be compared ?
Date Wed, 01 May 2002 15:05:27 GMT

Just to have some fun - Below is the result of running a build of AB
against the same apache 1.3.0 (stock)

It is a simple loop - checkout against a tag; cd apache-1.3/src && cp
Configuration.tmp && Configuration && ./Configure && make &&
cd support &&
make) and then run 20 times ./ab -c 30 -n 10000 and take the last 25 runs.

By the way - not a -single- tag failed to build on FreeBSD 5-Current and
not a single tag needed a differnet config ! A record hard to beat by
commercial software !

For each is displayed to total avaerage time/connection; the 98%
confidence interval (all in mSec) and the averegate for that version
of AB.

Note that typically (with eceptiosn early on) each version of AB produces
run within the confidence interval - and that successive versions their
confidence intertvals do generally not overlap (except early in the
development) in the more recent series.

So though this shows that comparing versions of AB from different versions
is generally waranted (same mean, same SD) - for recent versions of
1.3 - and that comparisions between version no's is not a good idea.

BUT at the same time - that has not always been the case - and that is is
kind of a miracle (or a tribute to release management/fact that AB hardly
ever changed) that comparison is possible. Also if you include 1.3c and
1.3a then you can conclude that these ranges match - thus from these
measurements you would conclude that the code did not change substantially
(it may of course do for keep alive or something).

Dw.

        Tag checout     #define in ab   mean/SD aver   CI
Tag:    APACHE_1_3_24   AB:     1.3d    66 +-3  66.5 +-3
Tag:    APACHE_1_3_23   AB:     1.3d    67 +-3
Tag:    APACHE_1_3_22   AB:     1.3d    67 +-3
Tag:    APACHE_1_3_21   AB:     1.3d    65 +-3

Tag:    APACHE_1_3_20   AB:     1.3c    63 +-2  61.8 +-2
Tag:    APACHE_1_3_19   AB:     1.3c    60 +-2
Tag:    APACHE_1_3_18   AB:     1.3c    61 +-2
Tag:    APACHE_1_3_17   AB:     1.3c    61 +-2
Tag:    APACHE_1_3_16   AB:     1.3c    63 +-2
Tag:    APACHE_1_3_15   AB:     1.3c    62 +-2
Tag:    APACHE_1_3_14   AB:     1.3c    63 +-2
Tag:    APACHE_1_3_13   AB:     1.3c    62 +-2
Tag:    APACHE_1_3_12   AB:     1.3c    62 +-2
Tag:    APACHE_1_3_11   AB:     1.3c    60 +-2
Tag:    APACHE_1_3_10   AB:     1.3c    63 +-2

Tag:    APACHE_1_3_9    AB:     1.3a    60 +-1  60.4 +-1
Tag:    APACHE_1_3_8    AB:     1.3a    61 +-1
Tag:    APACHE_1_3_7    AB:     1.3a    61 +-1

Tag:    APACHE_1_3_6    AB:     1.3     64 +-2  65.2 +-2
Tag:    APACHE_1_3_5    AB:     1.3     66 +-3

Tag:    APACHE_1_3_4    AB:     1.2     61 +-3  62.9 +-3
Tag:    APACHE_1_3_3    AB:     1.2     61 +-4
Tag:    APACHE_1_3_2    AB:     1.2     64 +-3

Tag:    APACHE_1_3_1    AB:     1.1     60 +-9  65.8 +-10
Tag:    APACHE_1_3_0    AB:     1.1     67 +-6
Tag:    APACHE_1_3b7    AB:     1.1     64 +-9
Tag:    APACHE_1_3b6    AB:     1.1     79 +-8

Dw
-- 
Dirk-Willem van Gulik



Mime
View raw message