tomcat-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roy Wilson <>
Subject Kudos to Jason and his benchmarking report
Date Wed, 06 Dec 2000 00:43:33 GMT

This is an impressive piece of work: very systematic and very 
well-documented. I have several questions. I ran some similar tests 
(without adjusting configuration parameters) and noticed variability as 
large as 10% in throughput that surprised me, even when N = 10,000. To 
reduce that, I felt compelled to use N = 100,000. Your data seems pretty 
stable: did you run more tests and just truncate the reporting? Do you 
think it would be worthwhile to rerun the test with N = 100,000 maybe two 
or three times ( my tests took hours )? 

Costin's ab data shows a ramp up: the shell script I posted a while back 
was based on that approach. I noticed doing my measurements that Apache 
throughput increased as C increased (up to C = 100), whereas TC 
throughput did not (actually it declined). I also wonder what the 
architectural explanation might be for the different scalability behavior 
(maybe this is obvious, I haven't thought about it yet). I wonder if you 
could (in your spare time :-)} repeat the test 3 times for C = 20, 40, 
60, 80, 100 with N = 100,000. That's my plan, along with using sar and sa 
with C = 1 to get CPU demand measures. Maybe others will join this little 
band of performance freaks. 

BTW, little credit should go to me re: ab use (or should I say "abuse"). 
Craig suggested using ab and Costin did it (and did it, ... :-)). Pier 
"inspired" me to look at ab code and think about how it can be used as a 
tool for getting information about both behavior and the resource usage 
that drives it. Your stuff (along with Costin's) might be a key part of a 
great guide to TCx performance and tuning ala Dean Gaudet's stuff on 
(Apache?) performance. Now I need to study your stuff more closely and 
plan for the next round of measurement (and, in my case, modeling).


Roy Wilson

>>>>>>>>>>>>>>>>>> Original Message

On 12/5/00, 1:47:46 PM, Jason Brittain <> wrote regarding 
Benchmarks: Catalina M5 vs. Apache 1.3.12:

> I wrote up a text file about benchmarking and comparing Tomcat-4.0-M5
> (pre-release)
> and Apache 1.3.12.  It's attached to this message.  I wrote it for
> anyone who is interested
> (even non-Java-saavy people) to know how the raw content serving
> performance of
> Catalina and its built-in web server compares to that of Apache 1.3.12.
> It contains lots of
> information to help people understand some of the important differences
> between the two
> servers.

> Feedback, flames, reproduced test results, etc. are welcome!  :)

> Cheers.

> --
> Jason Brittain
> Software Engineer, Olliance Inc.
> Current Maintainer, Locomotive Project

View raw message