jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Badeau, Kevin (DPH)" <Kevin.Bad...@state.ma.us>
Subject RE: Variances between automated and manual tests
Date Tue, 23 Jun 2009 15:39:37 GMT
I appreciate the response.

Point number 2 is one I've considered and would like to understand more of...
What exactly does it mean to say "jMeter is not a browser"?

One would be under the impression that jMeter is going through the same or similar interface
as a living breathing end user would as they used a browser but I questioned if that might
not be the case.

How then does one go about testing a 500 concurrent user scenario with each virtual user going
through all the various layers/applications a real user would be going through?

I attempted to use a text based browser to eliminate graphic rendering but the application
doesn't support a text based browser.

Kevin

-----Original Message-----
From: jmeter-user-return-25711-kevin.badeau=state.ma.us@jakarta.apache.org [mailto:jmeter-user-return-25711-kevin.badeau=state.ma.us@jakarta.apache.org]
On Behalf Of Andrew Melnyk
Sent: Tuesday, June 23, 2009 11:30 AM
To: JMeter Users List
Subject: Re: Variances between automated and manual tests

Hi Kevin,

There are several reasons why you can see such picture:

1.) Your jmeter scripts don't do what they intended to do. Make sure that
you have implemented in your tests assertions which guarantee that tests
doing right things.
2.) Jmeter not a browser so if there is a rendering of content on the client
side it does not measured by Jmeter.

Regards,
Andriy

2009/6/23 Badeau, Kevin (DPH) <Kevin.Badeau@state.ma.us>

> Hello folks,
>
>
>
> We are using jMeter to capacity test an application we are considering
> purchasing.
>
>
>
> When we ramp up to 500 concurrent users jMeter is reporting response times
> under 5 seconds and it appears it is stepping through all the functionality
> we are asking it to do.
>
>
>
> This is very acceptable for us.
>
>
>
> However, while the test is running we try to hit the application manually
> and we find it is unresponsive.
>
>
>
> Manual testing is quick outside of a concurrent jMeter test running.
>
> Manual testing performance degrades as we in range from 100 to 200
> concurrent users.
>
> Manual testing is unresponsive when we run 500 concurrent users.
>
>
>
> jMeter reports response times only degrade by about a ½ second for each
> level of concurrent users we try.
>
>
>
> There seems to be some wide performance variance from what jMeter is seeing
> vs. what we see manually.
>
>
>
> I'm wondering if anyone has any general suggestions as to why this might be
> or how we might go about isolating this anomaly.
>
>
>
> I can provide more specific details if needed but I feel the question is
> pretty basic in terms what we understand the tool is supposed to be
> accomplishing in simulating real world scenarios and benchmarking them.
>
>
>
> Thanks in advance.
>
>
>
> Kevin
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org


Mime
View raw message