jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deepak Shetty <shet...@gmail.com>
Subject Re: Variances between automated and manual tests
Date Tue, 23 Jun 2009 16:09:52 GMT
No rule of thumb  , I just watch the perf counters and see that the cpu load
is light to moderate and that memory doesn't exceed the physical ram on the
machine
regards
deepak

On Tue, Jun 23, 2009 at 9:02 AM, Scott McFadden
<SMcFadden@criticaltech.com>wrote:

> What is the recommended maximum number of simulated jmeter users / threads
> per machine?  Assuming Windows multi core environments.
>
> -----Original Message-----
> From: Deepak Shetty [mailto:shettyd@gmail.com]
> Sent: Tuesday, June 23, 2009 10:57 AM
> To: JMeter Users List
> Subject: Re: Variances between automated and manual tests
>
> hi
> Some things to think about
> a. Are you including all resources in your jmeter tests (e.g. all embedded
> resources like css/images/javascript which may or may not be cached by the
> browser).
> b. Do you have assertions for all your tests that validate that your
> response is as expected (no error messages , expected text etc?)
> c. Have you suitably parameterised your jmeter tests (say for e.g. using
> different users so that if your application is caching some bits per user ,
> then you dont get better results than you should)
> d. Have you distributed your tests across multiple machines correctly (500
> concurrent users wont normally be supported by a single client machine)?
> regards
> deepak
>
>
> On Tue, Jun 23, 2009 at 8:04 AM, Badeau, Kevin (DPH) <
> Kevin.Badeau@state.ma.us> wrote:
>
> > Hello folks,
> >
> >
> >
> > We are using jMeter to capacity test an application we are considering
> > purchasing.
> >
> >
> >
> > When we ramp up to 500 concurrent users jMeter is reporting response
> times
> > under 5 seconds and it appears it is stepping through all the
> functionality
> > we are asking it to do.
> >
> >
> >
> > This is very acceptable for us.
> >
> >
> >
> > However, while the test is running we try to hit the application manually
> > and we find it is unresponsive.
> >
> >
> >
> > Manual testing is quick outside of a concurrent jMeter test running.
> >
> > Manual testing performance degrades as we in range from 100 to 200
> > concurrent users.
> >
> > Manual testing is unresponsive when we run 500 concurrent users.
> >
> >
> >
> > jMeter reports response times only degrade by about a ½ second for each
> > level of concurrent users we try.
> >
> >
> >
> > There seems to be some wide performance variance from what jMeter is
> seeing
> > vs. what we see manually.
> >
> >
> >
> > I'm wondering if anyone has any general suggestions as to why this might
> be
> > or how we might go about isolating this anomaly.
> >
> >
> >
> > I can provide more specific details if needed but I feel the question is
> > pretty basic in terms what we understand the tool is supposed to be
> > accomplishing in simulating real world scenarios and benchmarking them.
> >
> >
> >
> > Thanks in advance.
> >
> >
> >
> > Kevin
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message