jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Melnyk <melnykena...@gmail.com>
Subject Re: Variances between automated and manual tests
Date Tue, 23 Jun 2009 15:30:11 GMT
Hi Kevin,

There are several reasons why you can see such picture:

1.) Your jmeter scripts don't do what they intended to do. Make sure that
you have implemented in your tests assertions which guarantee that tests
doing right things.
2.) Jmeter not a browser so if there is a rendering of content on the client
side it does not measured by Jmeter.

Regards,
Andriy

2009/6/23 Badeau, Kevin (DPH) <Kevin.Badeau@state.ma.us>

> Hello folks,
>
>
>
> We are using jMeter to capacity test an application we are considering
> purchasing.
>
>
>
> When we ramp up to 500 concurrent users jMeter is reporting response times
> under 5 seconds and it appears it is stepping through all the functionality
> we are asking it to do.
>
>
>
> This is very acceptable for us.
>
>
>
> However, while the test is running we try to hit the application manually
> and we find it is unresponsive.
>
>
>
> Manual testing is quick outside of a concurrent jMeter test running.
>
> Manual testing performance degrades as we in range from 100 to 200
> concurrent users.
>
> Manual testing is unresponsive when we run 500 concurrent users.
>
>
>
> jMeter reports response times only degrade by about a ½ second for each
> level of concurrent users we try.
>
>
>
> There seems to be some wide performance variance from what jMeter is seeing
> vs. what we see manually.
>
>
>
> I'm wondering if anyone has any general suggestions as to why this might be
> or how we might go about isolating this anomaly.
>
>
>
> I can provide more specific details if needed but I feel the question is
> pretty basic in terms what we understand the tool is supposed to be
> accomplishing in simulating real world scenarios and benchmarking them.
>
>
>
> Thanks in advance.
>
>
>
> Kevin
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message