jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: Increasing CPU during BOSH Test using JMETER
Date Sat, 05 Jun 2010 10:54:58 GMT
On 05/06/2010, Deepak Goel <deicool@gmail.com> wrote:
> Hey
>
>  Namaskara~Nalama~Guten Tag
>
>
> Sorry for the trouble.
>
>  Could you please post all the results: for the test servers and the presence
>  servers (Resource Utilizations, request sent over time, response got over
>  time, response time)? This would make it easier for us to find out the
>  problem.

It can be difficult reading logs etc posted inline, and attachments
are not always accepted by the list manager software, so ideally post
a link to the files you have uploaded to a public URL.

>  Thanks
>
> Deepak
>   --
>  Keigu
>
>  Deepak
>  +91-9765089593
>  deicool@gmail.com
>
>  Skype: thumsupdeicool
>  Google talk: deicool
>  Blog: http://loveandfearless.wordpress.com
>  Facebook: http://www.facebook.com/deicool
>
>  Check out my Work at:
>  LinkedIn: http://in.linkedin.com/in/thumsupdeicool
>
>  "Contribute to the world, environment and more : http://www.gridrepublic.org
>  "
>
>
>
> On Fri, Jun 4, 2010 at 9:41 PM, sebb <sebbaz@gmail.com> wrote:
>
>  > On 04/06/2010, frogg6 <fdunne2@yahoo.ie> wrote:
>  > >
>  > >  Yes, I am looking at the max CPU comsumption. It should not go over
>  > ~75%.
>  > >
>  > >  The high/low bursts may be explained by the way the script works. I have
>  > 4
>  > >  instances, each instance logs in 2500 users/threads, sends a burst of
>  > IMs,
>  > >  then sends a presence change, burst of IM's, presence change etc...
>  > >
>  > >  Nothing else is running on the jmeter boxes.
>  > >
>  > >  It is the latest OS version with up to dates patches.
>  > >
>  > >  It is in an isolated network.
>  > >
>  > >  Is jmeter capable of running such a test over 5 days?
>  > >
>  >
>  > Yes, provided that you take note of:
>  >
>  > http://jakarta.apache.org/jmeter/usermanual/best-practices.html#lean_mean
>  >
>  > However, I'm not convinced that this is a JMeter problem.
>  >
>  > In your original posting, you say:
>  >
>  > "... the presence server's CPU
>  > reaches about 90% which is a test failure. I have checked my presence
>  > server
>  > logs and everything looks fine so the problem must lie with the toolboxes."
>  >
>  > If I've understood this correctly, the JMeter systems are not the ones
>  > that are suffering from increasing CPU. Therefore the most likely
>  > problem is a resource leak or other problem in the code on the
>  > presence server.
>  >
>  > If the request load on the presence server is within its design
>  > limits, it should not be possible for an external source (such as
>  > JMeter) to cause the presence server to misbehave in this way.
>  >
>  > If the JMeter host is having problems running JMeter, then the load at
>  > the target server is likely to be reduced, not increased, and you
>  > won't be able to maintain the desired throughput. But that does not
>  > appear to be the problem here.
>  >
>  > >  Deepak Brijnandan Goel wrote:
>  > >  >
>  > >  > Hey
>  > >  >
>  > >  > Namaskara~Nalama~Guten Tag
>  > >  >
>  > >  > You already have transactions/sec in the reports (messages/sec,
>  > >  > presence/sec), which are maintained  at a constant rate.
>  > >  >
>  > >  > Also i checked the CPU consumed, it looks ok to me. You might need to
>  > run
>  > >  > for longer durations to check for abnormal behavior. The avg CPU rate
>  > has
>  > >  > increased by 10% over a days time (Are you looking at max CPU
>  > consumption
>  > >  > in
>  > >  > your reports?).
>  > >  >
>  > >  > Also the CPU consumption seems to drop in between (10000 to 14000) and
>  > >  > then
>  > >  > seems to increase (14000 to 17000)? This does suggest that the CPU
>  > >  > consumption due to the test might be ok, but there are some other
>  > threads
>  > >  > running on the test servers.
>  > >  >
>  > >  > Is there anything else running on the test servers? Like some
>  > maintenance
>  > >  > threads?
>  > >  >
>  > >  > Some other questions:
>  > >  >
>  > >  > 1. Is your system up to date with all OS patches? Is the latest OS
>  > system?
>  > >  > 2. Is it on a isolated network environment?
>  > >  > 3. We might also need to check whether Jmeter can run test at such
>  > high
>  > >  > rates for long duration of times (The test have been running for
>  > almost 2
>  > >  > days at a stretch). (Most of the memory leaks show up within a few
>  > hours.
>  > >  > One rarely needs to run test for such long times to check application
>  > >  > abnormality)
>  > >  >
>  > >  > Deepak
>  > >  >   --
>  > >  > Keigu
>  > >  >
>  > >  > Deepak
>  > >  > +91-9765089593
>  > >  > deicool@gmail.com
>  > >  >
>  > >  > Skype: thumsupdeicool
>  > >  > Google talk: deicool
>  > >  > Blog: http://loveandfearless.wordpress.com
>  > >  > Facebook: http://www.facebook.com/deicool
>  > >  >
>  > >  > Check out my Work at:
>  > >  > LinkedIn: http://in.linkedin.com/in/thumsupdeicool
>  > >  >
>  > >  > "Contribute to the world, environment and more :
>  > >  > http://www.gridrepublic.org
>  > >  > "
>  > >  >
>  > >  >
>  > >  > On Fri, Jun 4, 2010 at 7:12 PM, frogg6 <fdunne2@yahoo.ie> wrote:
>  > >  >
>  > >  >>
>  > >  >>  I sent you on my results. I will have to run a trace to check if
the
>  > >  >> transactions/sec is decreasing over time. If so, what would this
>  > suggest?
>  > >  >> How can it be fixed?
>  > >  >>
>  > >  >>
>  > >  >> Deepak Brijnandan Goel wrote:
>  > >  >> >
>  > >  >> > Hey
>  > >  >> >
>  > >  >> > Namaskara~Nalama~Guten Tag
>  > >  >> >
>  > >  >> > Could you please share the Jmeter test reports? Is the
>  > transactions/sec
>  > >  >> > decreasing and response time increasing over time?
>  > >  >> >
>  > >  >> > Deepak
>  > >  >> >   --
>  > >  >> > Keigu
>  > >  >> >
>  > >  >> > Deepak
>  > >  >> > +91-9765089593
>  > >  >> > deicool@gmail.com
>  > >  >> >
>  > >  >> > Skype: thumsupdeicool
>  > >  >> > Google talk: deicool
>  > >  >> > Blog: http://loveandfearless.wordpress.com
>  > >  >> > Facebook: http://www.facebook.com/deicool
>  > >  >> >
>  > >  >> > Check out my Work at:
>  > >  >> > LinkedIn: http://in.linkedin.com/in/thumsupdeicool
>  > >  >> >
>  > >  >> > "Contribute to the world, environment and more :
>  > >  >> > http://www.gridrepublic.org
>  > >  >> > "
>  > >  >> >
>  > >  >> >
>  > >  >> > On Fri, Jun 4, 2010 at 6:29 PM, frogg6 <fdunne2@yahoo.ie>
wrote:
>  > >  >> >
>  > >  >> >>
>  > >  >> >> Hi again,
>  > >  >> >>
>  > >  >> >> I don't think the problem lies with the presence sever
as
>  > everything
>  > >  >> >> looks
>  > >  >> >> ok on that side. It is something to do with the jmeter
toolboxes
>  > setup
>  > >  >> -
>  > >  >> >> possiblr the heap size/new size? I am not storing up IM
& presence
>  > >  >> >> requests,
>  > >  >> >> I simply fire them off to the presence server at a set
rate in a
>  > >  >> infinite
>  > >  >> >> loop once a user has logged in. I have logging enabled
to tell me
>  > if a
>  > >  >> >> request failed/timedout but everything looks ok. Right
now I am
>  > >  >> running
>  > >  >> 2
>  > >  >> >> instances of Jmeter [each of 1250 users] per toolbox. Would
it be
>  > a
>  > >  >> >> better
>  > >  >> >> idea to run 4 instances over 4 boxes?
>  > >  >> >>
>  > >  >> >>
>  > >  >> >> Deepak Brijnandan Goel wrote:
>  > >  >> >> >
>  > >  >> >> > Hey
>  > >  >> >> >
>  > >  >> >> > Namaskara~Nalama~Guten Tag
>  > >  >> >> >
>  > >  >> >> > This gets a bit tricky. How does your application
server the
>  > >  >> request?
>  > >  >> >> Does
>  > >  >> >> > it keep the history of all the request by each user
in the
>  > memory?
>  > >  >> What
>  > >  >> >> is
>  > >  >> >> > the caching mechanism? Are there any objects created
by each
>  > request
>  > >  >> >> which
>  > >  >> >> > is not released? Can profiling of the application
help in
>  > finding
>  > >  >> out
>  > >  >> >> > memory
>  > >  >> >> > leaks?
>  > >  >> >> >
>  > >  >> >> > Deepak
>  > >  >> >> >   --
>  > >  >> >> > Keigu
>  > >  >> >> >
>  > >  >> >> > Deepak
>  > >  >> >> > +91-9765089593
>  > >  >> >> > deicool@gmail.com
>  > >  >> >> >
>  > >  >> >> > Skype: thumsupdeicool
>  > >  >> >> > Google talk: deicool
>  > >  >> >> > Blog: http://loveandfearless.wordpress.com
>  > >  >> >> > Facebook: http://www.facebook.com/deicool
>  > >  >> >> >
>  > >  >> >> > Check out my Work at:
>  > >  >> >> > LinkedIn: http://in.linkedin.com/in/thumsupdeicool
>  > >  >> >> >
>  > >  >> >> > "Contribute to the world, environment and more :
>  > >  >> >> > http://www.gridrepublic.org
>  > >  >> >> > "
>  > >  >> >> >
>  > >  >> >> >
>  > >  >> >> > On Fri, Jun 4, 2010 at 5:20 PM, frogg6 <fdunne2@yahoo.ie>
>  > wrote:
>  > >  >> >> >
>  > >  >> >> >>
>  > >  >> >> >> Hey,
>  > >  >> >> >>
>  > >  >> >> >> Thanks for your quick reply! Once a user logs
in, he sends a
>  > >  >> constant
>  > >  >> >> >> rate
>  > >  >> >> >> of presence and IM to the presense server in a
infinite loop.
>  > In
>  > >  >> order
>  > >  >> >> to
>  > >  >> >> >> keep sending Presence and IM he will need to stay
logged in for
>  > the
>  > >  >> >> >> duration
>  > >  >> >> >> of the test. So in answer to your question, I
don't log out and
>  > >  >> users
>  > >  >> >> and
>  > >  >> >> >> their tasks never finish [until of course the
test is ended
>  > >  >> manually].
>  > >  >> >> >>
>  > >  >> >> >> Is it better to use a higher heap size? Can this
cause any
>  > >  >> negaitive
>  > >  >> >> >> effects?
>  > >  >> >> >>
>  > >  >> >> >>
>  > >  >> >> >>
>  > >  >> >> >> Deepak Brijnandan Goel wrote:
>  > >  >> >> >> >
>  > >  >> >> >> > Hey
>  > >  >> >> >> >
>  > >  >> >> >> > Namaskara~Nalama~Guten Tag
>  > >  >> >> >> >
>  > >  >> >> >> > You might like to check for memory leaks
in the application.
>  > Are
>  > >  >> >> your
>  > >  >> >> >> > users
>  > >  >> >> >> > logging out after finishing their task?
>  > >  >> >> >> >
>  > >  >> >> >> > You might also like to increase your max
heap size to 1 GB.
>  > >  >> >> >> >
>  > >  >> >> >> > Deepak
>  > >  >> >> >> >   --
>  > >  >> >> >> > Keigu
>  > >  >> >> >> >
>  > >  >> >> >> > Deepak
>  > >  >> >> >> > +91-9765089593
>  > >  >> >> >> > deicool@gmail.com
>  > >  >> >> >> >
>  > >  >> >> >> > Skype: thumsupdeicool
>  > >  >> >> >> > Google talk: deicool
>  > >  >> >> >> > Blog: http://loveandfearless.wordpress.com
>  > >  >> >> >> > Facebook: http://www.facebook.com/deicool
>  > >  >> >> >> >
>  > >  >> >> >> > Check out my Work at:
>  > >  >> >> >> > LinkedIn: http://in.linkedin.com/in/thumsupdeicool
>  > >  >> >> >> >
>  > >  >> >> >> > "Contribute to the world, environment and
more :
>  > >  >> >> >> > http://www.gridrepublic.org
>  > >  >> >> >> > "
>  > >  >> >> >> >
>  > >  >> >> >> >
>  > >  >> >> >> > On Fri, Jun 4, 2010 at 4:35 PM, frogg6 <fdunne2@yahoo.ie>
>  > wrote:
>  > >  >> >> >> >
>  > >  >> >> >> >>
>  > >  >> >> >> >> I am using Jmeter to generate BOSH login,
IM & Presence
>  > traffic
>  > >  >> to
>  > >  >> >> my
>  > >  >> >> >> >> presence server. The test is for 5000
users and the traffic
>  > from
>  > >  >> >> >> jmeter
>  > >  >> >> >> >> runs
>  > >  >> >> >> >> through a proxy server before going to
the presence server.
>  > I
>  > >  >> have
>  > >  >> >> >> setup
>  > >  >> >> >> >> JMeter on 2 servers running 2 identical
instances of my test
>  > >  >> >> scripts
>  > >  >> >> >> on
>  > >  >> >> >> >> each
>  > >  >> >> >> >> [2 x 2 x 1250 users/script = 5000 users].
The login period
>  > for
>  > >  >> 5000
>  > >  >> >> >> uers
>  > >  >> >> >> >> is
>  > >  >> >> >> >> 20 mins after which time each user generates
IM and Presence
>  > >  >> >> requests
>  > >  >> >> >> to
>  > >  >> >> >> >> the
>  > >  >> >> >> >> presence server. Everything is fine for
the first 24 hours,
>  > I
>  > >  >> see
>  > >  >> a
>  > >  >> >> >> nice
>  > >  >> >> >> >> steady CPU rate of around 60 % on my
presence server.
>  > However
>  > >  >> after
>  > >  >> >> 24
>  > >  >> >> >> >> hours
>  > >  >> >> >> >> the CPU gradually gets larger and larger
until the presence
>  > >  >> >> server's
>  > >  >> >> >> CPU
>  > >  >> >> >> >> reaches about 90% which is a test failure.
I have checked my
>  > >  >> >> presence
>  > >  >> >> >> >> server
>  > >  >> >> >> >> logs and everything looks fine so the
problem must lie with
>  > the
>  > >  >> >> >> >> toolboxes.
>  > >  >> >> >> >> Any idea what might be causing the CPU
to rise gradually
>  > after
>  > >  >> 24
>  > >  >> >> >> hours?
>  > >  >> >> >> >> Are
>  > >  >> >> >> >> there any memory configurations etc that
need to be set
>  > prior to
>  > >  >> >> >> running
>  > >  >> >> >> >> a
>  > >  >> >> >> >> test? I am using a HeapSize of min: 256,
max: 512 which for
>  > each
>  > >  >> >> >> >> instance.
>  > >  >> >> >> >> Do the NewSize values have to match these
in terms of ratio?
>  > My
>  > >  >> >> >> toolboxes
>  > >  >> >> >> >> also contain 4 processors so I am tying
each jmeter instance
>  > to
>  > >  >> a
>  > >  >> >> >> >> separate
>  > >  >> >> >> >> processor which seems to fix the problem
intermittently. Any
>  > >  >> help
>  > >  >> >> >> would
>  > >  >> >> >> >> be
>  > >  >> >> >> >> much appreciated :-)
>  > >  >> >> >> >> --
>  > >  >> >> >> >> View this message in context:
>  > >  >> >> >> >>
>  > >  >> >> >>
>  > >  >> >>
>  > >  >>
>  > http://old.nabble.com/Increasing-CPU-during-BOSH-Test-using-JMETER-tp28778525p28778525.html
>  > >  >> >> >> >> Sent from the JMeter - User mailing list
archive at
>  > Nabble.com.
>  > >  >> >> >> >>
>  > >  >> >> >> >>
>  > >  >> >> >> >>
>  > >  >> >>
>  > ---------------------------------------------------------------------
>  > >  >> >> >> >> To unsubscribe, e-mail:
>  > >  >> jmeter-user-unsubscribe@jakarta.apache.org
>  > >  >> >> >> >> For additional commands, e-mail:
>  > >  >> >> jmeter-user-help@jakarta.apache.org
>  > >  >> >> >> >>
>  > >  >> >> >> >>
>  > >  >> >> >> >
>  > >  >> >> >> >
>  > >  >> >> >>
>  > >  >> >> >> --
>  > >  >> >> >> View this message in context:
>  > >  >> >> >>
>  > >  >> >>
>  > >  >>
>  > http://old.nabble.com/Increasing-CPU-during-BOSH-Test-using-JMETER-tp28778525p28778903.html
>  > >  >> >> >> Sent from the JMeter - User mailing list archive
at Nabble.com.
>  > >  >> >> >>
>  > >  >> >> >>
>  > >  >> >> >>
>  > >  >> ---------------------------------------------------------------------
>  > >  >> >> >> To unsubscribe, e-mail:
>  > jmeter-user-unsubscribe@jakarta.apache.org
>  > >  >> >> >> For additional commands, e-mail:
>  > >  >> jmeter-user-help@jakarta.apache.org
>  > >  >> >> >>
>  > >  >> >> >>
>  > >  >> >> >
>  > >  >> >> >
>  > >  >> >>
>  > >  >> >> --
>  > >  >> >> View this message in context:
>  > >  >> >>
>  > >  >>
>  > http://old.nabble.com/Increasing-CPU-during-BOSH-Test-using-JMETER-tp28778525p28779651.html
>  > >  >> >> Sent from the JMeter - User mailing list archive at Nabble.com.
>  > >  >> >>
>  > >  >> >>
>  > >  >> >>
>  > ---------------------------------------------------------------------
>  > >  >> >> To unsubscribe, e-mail:
>  > jmeter-user-unsubscribe@jakarta.apache.org
>  > >  >> >> For additional commands, e-mail:
>  > jmeter-user-help@jakarta.apache.org
>  > >  >> >>
>  > >  >> >>
>  > >  >> >
>  > >  >> >
>  > >  >>
>  > >  >> --
>  > >  >> View this message in context:
>  > >  >>
>  > http://old.nabble.com/Increasing-CPU-during-BOSH-Test-using-JMETER-tp28778525p28780112.html
>  > >  >> Sent from the JMeter - User mailing list archive at Nabble.com.
>  > >  >>
>  > >  >>
>  > >  >> ---------------------------------------------------------------------
>  > >  >> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
>  > >  >> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>  > >  >>
>  > >  >>
>  > >  >
>  > >  >
>  > >
>  > >
>  > > --
>  > >  View this message in context:
>  > http://old.nabble.com/Increasing-CPU-during-BOSH-Test-using-JMETER-tp28778525p28781903.html
>  > >
>  > > Sent from the JMeter - User mailing list archive at Nabble.com.
>  > >
>  > >
>  > >  ---------------------------------------------------------------------
>  > >  To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
>  > >  For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>  > >
>  > >
>  >
>  > ---------------------------------------------------------------------
>  > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
>  > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>  >
>  >
>

---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org


Mime
View raw message