jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robin D. Wilson" <rwils...@gmail.com>
Subject RE: Average times are confusing me...
Date Fri, 10 Feb 2012 19:56:37 GMT
OK, I just took your exact same 'jsp', removed the RandomStringUtils stuff (I don't have that
library handy), and I'm still getting
the same weird results.

I'll try to package up my test case today and make it available to everyone...

--
Robin D. Wilson
Sr. Director of Web Development
KingsIsle Entertainment, Inc.
VOICE: 512-777-1861
www.KingsIsle.com


-----Original Message-----
From: Philippe Mouawad [mailto:philippe.mouawad@gmail.com] 
Sent: Friday, February 10, 2012 7:41 AM
To: JMeter Users List
Subject: Re: Average times are confusing me...

Hello,
I have just made the same test with a JSP (doing a Thread.sleep(2000) =>
2s) behind a Tomcat 6 with JMeter 2.6

I put 100 threads that loop 100 times in GUI mode (one JMETER):
All results shown in Aggregate report are around 2s .










sampler_label,aggregate_report_count,average,aggregate_report_median,aggregate_report_90%_line,aggregate_report_min,aggregate_report
_max,aggregate_report_error%,aggregate_report_rate,aggregate_report_bandwidth
HTTP
Request,10000,2006,2004,2012,2001,2195,0.0,49.5793194742609,1224.082013722936
TOTAL,10000,2006,2004,2012,2001,2195,0.0,49.5793194742609,1224.082013722936


Summary listener shows this:
2012/02/10 14:36:00 INFO  - jmeter.reporters.Summariser: Generate Summary
Results +  2655 in  54,8s =   48,4/s Avg:  2010 Min:  2001 Max:  2195
Err:     0 (0,00%)
2012/02/10 14:38:26 INFO  - jmeter.reporters.Summariser: Generate Summary
Results +  7345 in 148,9s =   49,3/s Avg:  2005 Min:  2001 Max:  2088
Err:     0 (0,00%)
2012/02/10 14:38:26 INFO  - jmeter.reporters.Summariser: Generate Summary
Results = 10000 in 201,7s =   49,6/s Avg:  2006 Min:  2001 Max:  2195
Err:     0 (0,00%)

So as you can see, nothing abnormal .

JSP Code:

<%@ page  contentType="text/html; charset=UTF-8" %>
<html>
<body>
<%
Thread.sleep(2000);
%>
<!-- GENERATE around 25Ko page -->
<%=org.apache.commons.lang.RandomStringUtils.random(25000,
"abcdefghijklmnopqrstuvwxyz01234567890")%>
<p>Test</p>
</body>
</html>


Regards
Philippe
http://www.ubik-ingenierie.com

On Thu, Feb 9, 2012 at 4:52 PM, sebb <sebbaz@gmail.com> wrote:

> On 9 February 2012 15:38, Robin D. Wilson <rwilson2@gmail.com> wrote:
> > Thanks sebb for the replies...
> >
> > Here's the deal, I am running the same test script on JM2.4 and JM2.6. I
> am running in GUI mode. The test script has 3 thread groups
> > - but the first and the last thread group is just a 'timer' I created to
> log the total elapsed time of the test (the first and last
> > group has 1 thread, and 1 request, and take less than 1 second each to
> run). The 'real' test is the middle thread group. It has 100
> > threads (0 ramp), and runs 100 iterations (10,000 total samples). It
> simply does a 'POST' to a URL, with 15
> >
> > So the 'elapsed time' I referring to in my test is actually the
> timestamp taken in the first thread group (in ms since epoch)
> > subtracted from the timestamp taken in the 3rd (last) thread group. That
> part of my test may only add 2 total seconds to the test,
> > so while it may skew my results slightly - it doesn't explain the vast
> difference in the 'average' sample duration. According to the
> > Summary Report docs, the "Average" is supposed to be "the average
> elapsed time of a set of samples". But clearly, if the minimum
> > time it takes to actually get the page is 2 seconds (due to the built-in
> delay in the cgi-script), there is no way I could have an
> > 'average' elapsed time of less than 2 seconds, yet I'm showing an
> average elapsed time of ~750 ms... (My "Max" elapsed time shows as
> > only 1198!). When I request the page in Firefox, it takes ~2104ms (using
> a status bar timer), so I think the cgi script is working
> > correctly.)
> >
> > Sebb asked:
> >
> >>Again, the throughput calculations are based on total test time. Are you
> sure the test run times are comparable?
> >
> > The test run times are automatically calculated by the 1st and 3rd
> thread groups. The ~210 seconds total elapsed time is accurate
> > based on my external measurement too (e.g., it is close to what I can
> observe with my stopwatch).
> >
> > Both the JM2.4 test and the JM2.6 test are using the exact same ".jmx"
> test file.
> >
> >>There's clearly something else going on here.
> >
> > I don't believe that the Summary Report is accurately calculating
> anything except the total number of samples and the Avg. Bytes...
>
> What makes you say that?
> Are the Min and Max really incorrect?
> Error %?
>
> It's easy enough to check the Summary Results if you can provide the
> CSV sample result files.
>
> > The cgi-script I'm using definitely takes 2+ seconds to respond after it
> gets the request (I've measured this with Firefox directly,
> > and it _never_ gets a response in less than 2 seconds). I even changed
> the 'sleep' to 9 seconds, and JMeter pauses for that long in
> > recording results (e.g., it shows 100 threads run, then waits 9 seconds,
> shows another 100 threads, etc.), but the numbers just go
> > up to '1758' Average, and '2415' Max (which is impossible since it is
> taking 9+ seconds to respond to each request!). It takes over
> > 15 minutes to complete 10,000 samples (and that seems about right -
> 10000 samples/100 threads * 9 seconds each = 900 seconds).
> >
> > I even went so far as to inject a 2 second sleep in the middle of the
> response (e.g., pause 2 seconds -  send part of the response -
> > pause 2 more seconds - send the rest), I'm still getting average times
> of ~1000 ms. (That's with 4 seconds of built-in delays, and 2
> > of those seconds are in the middle of the response.) The browser shows
> this delay properly, but JMeter isn't calculating it
> > properly.
> >
> >>Please recheck the individual sample response times and see how they
> compare to the average.
> >
> > I'm not sure how to do that in JMeter. I can manually hit the page, and
> it takes about 100ms longer than the built-in delay I have.
>
> Add a Table View Listener, or just check the CSV sample result files.
>
> >>If there still appears to be a problem, create a Bugzilla issue and
> attach:
> >>- JMX test case
> >
> > I'm trying to simplify the test case to the bare minimum case - so the
> results will be indisputable. I will also include the
> > 'cgi-bin' script that I'm using, so someone else can easily setup the
> same test.
>
> Thanks.
>
> >
> >>- log files for JMeter 2.4 and 2.6
> >
> > Which log files are these? Is it just the 'jmeter.log' that gets created
> in the 'bin' folder when I run the GUI mode, or do you need
> > another log file?
>
> jmeter.log
>
> >>- CSV result files for 2.4 and 2.6
> >
> > I can do this.
> >
> > --
> > Robin D. Wilson
> > Sr. Director of Web Development
> > KingsIsle Entertainment, Inc.
> > VOICE: 512-777-1861
> > www.KingsIsle.com
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> > For additional commands, e-mail: user-help@jmeter.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>


-- 
Cordialement.
Philippe Mouawad.


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org


Mime
View raw message