jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Stover <mstov...@apache.org>
Subject Re: Any other limitations of jmeter? How about scalability?
Date Thu, 15 Jul 2004 18:07:08 GMT
I have often wondered if removing timers is an efficient way of
simulating more users than otherwise possible.  Ie, if you remove
timers, 4 threads is equivalent to X threads delayed 2 seconds between
requests.  In terms of load generated on the server.  The elimination of
extra threads probably helps reduce some of the context switching and
memory use.

Then again, no timers might increasing context switching, so who knows?

-Mike

On Thu, 2004-07-15 at 13:49, Peter Lin wrote:
> I should mention that in my case, I am brutal. I use ramp up, but I
> don't use timers. I tend to shoot for worse case scenario to expose
> flaws, weaknesses and memory leaks in the dev and staging environment.
> 
> I usually try to shoot for better performance than the requirements.
> If my stress tests handle the worse case test, I usually feel
> confortable standing by the performance of the application. If it just
> barely meets the requirements, I usually state a lower number for
> sanity sake.
> 
> I might be paranoid.
> 
> peter
> 
> 
> On Thu, 15 Jul 2004 13:51:57 -0400, Michael Stover <mstover1@apache.org> wrote:
> > If you are on 3Ghz p4 with 1G, I would expect you can run more than 100
> > users.  I've run up to 500 with reasonable success (you can't trust the
> > absolute timings on it, but it doesn't chug badly or crash).
> > 
> > However, it's all highly dependent on the test details.  A test that
> > parses responses for embedded resources is going to kill JMeter's
> > performance a lot faster than one that doesn't.  Regular expression
> > parsers, assertions also add significant processing time for JMeter.  A
> > complex test might have 100 users as a max on that machine.  A simple
> > test might allow up to 500 users.
> > 
> > It also depends on how fast the server is - a faster server puts more
> > stress on JMeter because each thread spends less time blocked on IO and
> > more time preparing the next request.
> > 
> > And, of course, timers.  A test without timers is brutal for both
> > sides.  A few timers increases dramatically the number of users that can
> > effectively be simulated.
> > 
> > I think the biggest lingering limitation of JMeter is that the remote
> > testing feature was not adequately made, and does not in fact get you
> > anything in terms of scaling tests up.
> > 
> > -Mike
> > 
> > 
> > 
> > On Thu, 2004-07-15 at 13:05, joelsherriff wrote:
> > > Agree - my background is in writing commercial test tools - was team lead
> > > for Compuware on the web team for QALoad, before that with
> > > Rational/Pure/Performix (way way back when).  So the big tests 1000+ users
> > > are what I'm used to dealing with.  Right now I've been asked to evaluate
> > > jmeter against the commercial tools and scalability is always the killer -
> > > esp when you look at the tools that drive the server by driving instances of
> > > a browser (yuck!).  I'm really liking jmeter - much better than I expected
> > > it to be.  I expected to find lot's of holes and limitations (the arrogance
> > > of the old-timer) but really haven't seen any that aren't there for pretty
> > > good reason - like the proxy/ssl issue.
> > >
> > > To distill down something you said, and ignoring the network limitations
> > > (since that's such a variable):  so in your experience you get about 100
> > > threads per machine before it saturates it?  Is it typically CPU bound or
> > > memory or...?
> > >
> > > J
> > >
> > > ----- Original Message -----
> > > From: "Peter Lin" <woolfel@gmail.com>
> > > To: "JMeter Users List" <jmeter-user@jakarta.apache.org>
> > > Sent: Thursday, July 15, 2004 11:43 AM
> > > Subject: Re: Any other limitations of jmeter? How about scalability?
> > >
> > >
> > > > for general performance testing stuff, I have several performance
> > > > articles listed on the links page
> > > > http://wiki.apache.org/jakarta-jmeter/JMeterLinks.
> > > >
> > > > If I compare JMeter to other tools I've used, I would say it is
> > > > comparable. The tools I've used:
> > > >
> > > > 1. apache ab
> > > > 2. zdnet Webbench
> > > > 3. custom script perl
> > > >
> > > > The biggest challenge of stress testing is this, "what are you
> > > > testing?"  Most of the time, the bottleneck for a website is the
> > > > connection to the internet. I have a explanation of it in the
> > > > articles. It doesn't really matter if your website can handle 80
> > > > concurrent requests reliably, if you're server is on a T1. I would say
> > > > JMeter can easily produce enough stress to figure out what the upper
> > > > limit is for your web application. The situations where it won't be
> > > > suitable is if you're testing a system that has to support 5,000
> > > > concurrent users. You still could use JMeter if you setup each
> > > > instance with 100 threads and use 50 machines.
> > > >
> > > > realistically, to do that kind of test, the server would have to be a
> > > > big Unix box or mainframe hooked up to several gigabit routers. Using
> > > > 100mbit ethernet, you wouldn't be able to test 5K concurrent requests
> > > > for a sustained period of time effectively. Sustained transfer rate
> > > > plays a huge role in how many concurrent requests a system can handle
> > > > reliably for a long time.
> > > >
> > > > peter
> > > >
> > > >
> > > >
> > > > On Thu, 15 Jul 2004 11:25:50 -0400, joelsherriff
> > > > <joelsherriff@comcast.net> wrote:
> > > > > After Peter pointed out that the jmeter proxy doesn't do ssl, I thought
> > > I'd
> > > > > ask everyone if they know of other limitations they've run into and
> > > about
> > > > > what seems to be the cronic limitation of these tools - scalability.
> > > Not
> > > > > talking about bugs per se, just things you might have been able to
do
> > > with
> > > > > other tools you can't do with jmeter.  Like, does jmeter support
NTLM
> > > > > authentication?  (I realize that's a question, not a statement, but
some
> > > > > other tools have problems with NTLM so I thought I'd throw it out
> > > there.)
> > > > >
> > > > > As to scalability...How many users can you emulate before saturating
> > > your
> > > > > driver machine given an "average" script (mix of text and graphics,
> > > etc)?
> > > > > Given something like a 3Ghz P4 with 1M of memory.
> > > > >
> > > > > I originally did intend to play with jmeter more and lurk on this
list
> > > much
> > > > > longer before asking these types of questions, but, as always, it's
> > > faster
> > > > > to ask...
> > > > >
> > > > > J
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > > > > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> > > >
> > > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> > --
> > Michael Stover <mstover1@apache.org>
> > Apache Software Foundation
> > 
> > 
> > 
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> > 
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
-- 
Michael Stover <mstover1@apache.org>
Apache Software Foundation


---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org


Mime
View raw message