jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bo Bodnar" <bbod...@us.ibm.com>
Subject Re: Best way to compare two results of jmeter
Date Thu, 30 Aug 2018 14:35:39 GMT

I've been away from jmeter for a while.  Working from memory:

The scripts I used had functional breakdown with the topic of interest
being latency.  Each functional area had latency samples associated with
it.  For example "login" for run1 would have X samples and the same was for
run2.  It is important that measurement conditions be identical for each
run; otherwise, the performance of interest may be affected  by
uncontrolled conditions (e.g., time-dependent internet delays).  It is also
VERY important that the system of interest be in exactly the same condition
when the runs are done; if not, there may be memory impacts.  I then did
paired t-testing to see if the runs were statistically similar (I think I
used a confidence interval of 0.95).  X should be greater than around 10;
otherwise, there may be too much variability in the measurements and you
may have false conclusions.

                                                                                     
   Bohdan L. Bodnar                                                                  
   Lead Performance Engineer                                                         
   1-312-871-5163                                                                    
                                                                                     
                                                                                      
                                                                                      
                                                                                      
   E-mail: bbodnar@us.ibm.com                               222 South Riverside Plaza 
                                                                    Chicago, IL 60606 
                                                                        United States 
                                                                                      





From:	Marek Czernek <mczernek@redhat.com>
To:	user@jmeter.apache.org
Date:	08/30/2018 09:11 AM
Subject:	Best way to compare two results of jmeter



Hi there,

is there any 'supported' way to compare the results of 2 jmeter runs? I
googled around and found an old email from 2004 [1] basically saying
that there is no recommended solution other than a custom-made analysis.
Have there been any solutions to this problem?

I can also see a Jenkins plugin [2] though I have no idea in what state
the plugin is, and as such, how viable it is to run it. Last but not
least, there's some Grafana integration blog [3].  Does anyone have any
other suggestions? How would you compare two results programmatically to
see if there is degraded performance?

[1]
http://mail-archives.apache.org/mod_mbox/jmeter-user/200401.mbox/%3C99805B014A26D211BC3100A0C9B72CB8059E28EA@exchange-va.noblestar.com%3E


[2]
https://wiki.jenkins.io/display/JENKINS/Performance+Plugin


[3]
http://www.testautomationguru.com/jmeter-real-time-results-influxdb-grafana/


Cheers,
--

Marek Czernek

JWS/JBCS Associate Quality Engineer, RHCA

Find me at www.halfastack.com




Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message