Mailing-List: contact dev-help@geronimo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@geronimo.apache.org
Received-SPF: pass (asf.osuosl.org: local policy)
Message-ID: <44340026.6080600@hogstrom.org>
Date: Wed, 05 Apr 2006 13:36:38 -0400
From: Matt Hogstrom <matt@hogstrom.org>
User-Agent: Mozilla Thunderbird 1.0.7 (Macintosh/20050923)
MIME-Version: 1.0
To: dev@geronimo.apache.org
Subject: Re: Geronimo peak performance
References: <eac5ea1a0604050619i2778bb93q4bb5d4f013e8291a@mail.gmail.com>
In-Reply-To: <eac5ea1a0604050619i2778bb93q4bb5d4f013e8291a@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Maxim,

Thanks for sharing your results.  I have a whole set of numbers that I've been sitting on.  My tests 
don't use JMeter so if you would like to share your setup for that I'll incorporate it into the 
DayTrader tree.

All the tests I've run were with a fixed number of users of 100 with 10ms think time.  My goal was 
to stress the server and see if it would stay up and how it would perform.  The system I'm testing 
on is an Intel 2 x 3.0Ghz Potomac System.  Each processor has 8MB L3 Cache.  My tests were conducted 
with the Sun JDK (1.4.2_b09).  The Database system is on a separate box.  I'm using DB2 as Oracle 
has some clause in their license that does not allow publish of benchmark results without their 
express permission.

See inline

I'm rerunning some tests this afternoon as it looks like we're not comparing teh same things.  I am 
using an internal Load Generator and would like to move something Open Source so we can all compare 
the same numbers.

Matt

Maxim Berkultsev wrote:
> Hi, all!
> 
> Geronimo peak performance is under test and let me share some results.
> 
> I was using JMeter and Daytrader web primitives to measure throughput
> for a fixed number of simultaneously woking virtual users.
> 
> However I've realized that the results for the equal numbers of users
> in each scenario do not look valuable and tried to find some peak
> values for throughput depending on the number of users. It looks as if
> such peak values are reached when the number of users are minimal as
> well as the peak exists at some 'optimal' number of users.
> 
> I've used two scenarios.
> 
> Example 1: For scenario PingServlet2SessionEJB from Daytrader web
> primitives I've got max througput (~14670) for a minimal number of
> users - 5 with average time per single request equals to 17. The table
> below contains triples (number of users, average request time,
> throughput) for different number of users.
> 
> ----------------
> 5 17 14670
> ----------------
> 10 40 13037
> ----------------
> 50 188 12646
> ----------------
> 100 447 11028
   100 168 582   << I'm confused by this.  Actually all these numbers are way higher than I'm 
achieving.  Can you shed some light on your configuration?  Also, I assume your not getting 404's or 
something?
> ----------------
> 150 588 10770
> ----------------
> 200 634 10444
> ----------------
> 
> It looks as if the peak is reached when a number of users is minimal.
> 
> Example 2: In scenario PingServlet2TwoPhase the throughput grows to
> some saturation value and then begins to decrease. The maximal values
> (~1300-1350) for throughput covers a wide interval between >10 and 150
> virtual users. Here are the triples' table (number of users, average
> request time, throughput):
> 
> ----------------
> 5, 390, 764
> ----------------
> 10, 492, 1207
> ----------------
> 50, 2250, 1314
> ----------------
> 100, 4380, 1356
   100  2096    46  << Again...something is out of sorts.  Can you run Direct Mode with 5000 Users 
and 10000 quotes?
> ----------------
> 150, 6580, 1350
> ----------------
> 200, 9050, 1260
> ----------------
> 
> All values do not pretend to a significant:) accuracy but to some general trend.
> 
> Somehow usually there is some 'common sense' number of users to be
> used in performance estimations. Can someone provide an idea how to
> find this value for Geronimo?
> 
> Thank you.
> 
> --
> Best regards,
> Maxim Berkultsev, Intel Middleware Products Division
> 
> 
>