jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Stover <mstov...@apache.org>
Subject Re: Any other limitations of jmeter? How about scalability?
Date Thu, 15 Jul 2004 17:51:57 GMT
If you are on 3Ghz p4 with 1G, I would expect you can run more than 100
users.  I've run up to 500 with reasonable success (you can't trust the
absolute timings on it, but it doesn't chug badly or crash).  

However, it's all highly dependent on the test details.  A test that
parses responses for embedded resources is going to kill JMeter's
performance a lot faster than one that doesn't.  Regular expression
parsers, assertions also add significant processing time for JMeter.  A
complex test might have 100 users as a max on that machine.  A simple
test might allow up to 500 users.  

It also depends on how fast the server is - a faster server puts more
stress on JMeter because each thread spends less time blocked on IO and
more time preparing the next request.

And, of course, timers.  A test without timers is brutal for both
sides.  A few timers increases dramatically the number of users that can
effectively be simulated.

I think the biggest lingering limitation of JMeter is that the remote
testing feature was not adequately made, and does not in fact get you
anything in terms of scaling tests up.

-Mike

On Thu, 2004-07-15 at 13:05, joelsherriff wrote:
> Agree - my background is in writing commercial test tools - was team lead
> for Compuware on the web team for QALoad, before that with
> Rational/Pure/Performix (way way back when).  So the big tests 1000+ users
> are what I'm used to dealing with.  Right now I've been asked to evaluate
> jmeter against the commercial tools and scalability is always the killer -
> esp when you look at the tools that drive the server by driving instances of
> a browser (yuck!).  I'm really liking jmeter - much better than I expected
> it to be.  I expected to find lot's of holes and limitations (the arrogance
> of the old-timer) but really haven't seen any that aren't there for pretty
> good reason - like the proxy/ssl issue.
> 
> To distill down something you said, and ignoring the network limitations
> (since that's such a variable):  so in your experience you get about 100
> threads per machine before it saturates it?  Is it typically CPU bound or
> memory or...?
> 
> J
> 
> ----- Original Message ----- 
> From: "Peter Lin" <woolfel@gmail.com>
> To: "JMeter Users List" <jmeter-user@jakarta.apache.org>
> Sent: Thursday, July 15, 2004 11:43 AM
> Subject: Re: Any other limitations of jmeter? How about scalability?
> 
> 
> > for general performance testing stuff, I have several performance
> > articles listed on the links page
> > http://wiki.apache.org/jakarta-jmeter/JMeterLinks.
> >
> > If I compare JMeter to other tools I've used, I would say it is
> > comparable. The tools I've used:
> >
> > 1. apache ab
> > 2. zdnet Webbench
> > 3. custom script perl
> >
> > The biggest challenge of stress testing is this, "what are you
> > testing?"  Most of the time, the bottleneck for a website is the
> > connection to the internet. I have a explanation of it in the
> > articles. It doesn't really matter if your website can handle 80
> > concurrent requests reliably, if you're server is on a T1. I would say
> > JMeter can easily produce enough stress to figure out what the upper
> > limit is for your web application. The situations where it won't be
> > suitable is if you're testing a system that has to support 5,000
> > concurrent users. You still could use JMeter if you setup each
> > instance with 100 threads and use 50 machines.
> >
> > realistically, to do that kind of test, the server would have to be a
> > big Unix box or mainframe hooked up to several gigabit routers. Using
> > 100mbit ethernet, you wouldn't be able to test 5K concurrent requests
> > for a sustained period of time effectively. Sustained transfer rate
> > plays a huge role in how many concurrent requests a system can handle
> > reliably for a long time.
> >
> > peter
> >
> >
> >
> > On Thu, 15 Jul 2004 11:25:50 -0400, joelsherriff
> > <joelsherriff@comcast.net> wrote:
> > > After Peter pointed out that the jmeter proxy doesn't do ssl, I thought
> I'd
> > > ask everyone if they know of other limitations they've run into and
> about
> > > what seems to be the cronic limitation of these tools - scalability.
> Not
> > > talking about bugs per se, just things you might have been able to do
> with
> > > other tools you can't do with jmeter.  Like, does jmeter support NTLM
> > > authentication?  (I realize that's a question, not a statement, but some
> > > other tools have problems with NTLM so I thought I'd throw it out
> there.)
> > >
> > > As to scalability...How many users can you emulate before saturating
> your
> > > driver machine given an "average" script (mix of text and graphics,
> etc)?
> > > Given something like a 3Ghz P4 with 1M of memory.
> > >
> > > I originally did intend to play with jmeter more and lurk on this list
> much
> > > longer before asking these types of questions, but, as always, it's
> faster
> > > to ask...
> > >
> > > J
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> >
> >
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
-- 
Michael Stover <mstover1@apache.org>
Apache Software Foundation


---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org


Mime
View raw message