jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bo Bodnar" <>
Subject Re: Best way to compare two results of jmeter
Date Thu, 30 Aug 2018 14:35:39 GMT

I've been away from jmeter for a while.  Working from memory:

The scripts I used had functional breakdown with the topic of interest
being latency.  Each functional area had latency samples associated with
it.  For example "login" for run1 would have X samples and the same was for
run2.  It is important that measurement conditions be identical for each
run; otherwise, the performance of interest may be affected  by
uncontrolled conditions (e.g., time-dependent internet delays).  It is also
VERY important that the system of interest be in exactly the same condition
when the runs are done; if not, there may be memory impacts.  I then did
paired t-testing to see if the runs were statistically similar (I think I
used a confidence interval of 0.95).  X should be greater than around 10;
otherwise, there may be too much variability in the measurements and you
may have false conclusions.

   Bohdan L. Bodnar                                                                  
   Lead Performance Engineer                                                         
   E-mail:                               222 South Riverside Plaza 
                                                                    Chicago, IL 60606 
                                                                        United States 

From:	Marek Czernek <>
Date:	08/30/2018 09:11 AM
Subject:	Best way to compare two results of jmeter

Hi there,

is there any 'supported' way to compare the results of 2 jmeter runs? I
googled around and found an old email from 2004 [1] basically saying
that there is no recommended solution other than a custom-made analysis.
Have there been any solutions to this problem?

I can also see a Jenkins plugin [2] though I have no idea in what state
the plugin is, and as such, how viable it is to run it. Last but not
least, there's some Grafana integration blog [3].  Does anyone have any
other suggestions? How would you compare two results programmatically to
see if there is degraded performance?





Marek Czernek

JWS/JBCS Associate Quality Engineer, RHCA

Find me at

  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message