jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tuukka Mustonen <tuukka.musto...@gmail.com>
Subject Re: High CPU utilization in JMeter 3.x with HttpClient 4 leads to freeze
Date Wed, 01 Feb 2017 12:26:55 GMT
*dump_20170201_124717_708.hprof* is the file that JMeter created. That
should be in the linked folder...

Tuukka

Tuukka


On Wed, Feb 1, 2017 at 2:09 PM, Philippe Mouawad <philippe.mouawad@gmail.com
> wrote:

> Hi,
> I don't see any heap dump.
> When JMeter creates it it shows the file name.
> Regards
>
> On Wed, Feb 1, 2017 at 12:57 PM, Tuukka Mustonen <
> tuukka.mustonen@gmail.com>
> wrote:
>
> > Ok I re-ran the test.
> >
> > Took Heap Dump just before JMeter froze.
> >
> > Took Thread Dumps right after JMeter froze.
> >
> > https://drive.google.com/drive/folders/0B1uBdVuLED5NNFNqallGam13MTA
> >
> > ...
> >
> > For the fun of it, I also tried upgrading to the latest HttpClient 4.5.3
> > version. No effect.
> >
> > I also noticed that if I ping (google) during the perf testing, Windows
> > ping reports "General failure." when JMeter breaks. After killing it,
> > connectivity restores and there are replies from the server again. It
> seems
> > like JMeter/HttpClient kills my whole network connection...
> >
> > Tuukka
> >
> >
> > On Wed, Feb 1, 2017 at 11:44 AM, Philippe Mouawad <
> > philippe.mouawad@gmail.com> wrote:
> >
> > > On Wed, Feb 1, 2017 at 9:55 AM, Tuukka Mustonen <
> > tuukka.mustonen@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hi, I'm running solely on Win 10 now. In re-attempts I think I saw
> > > slightly
> > > > different behavior in my Linux guest. I'll write more about that
> later,
> > > but
> > > > for simplicity I'm testing only on Windows host now:
> > > >
> > > > Thread dumps are downloadable at:
> > > > https://drive.google.com/drive/folders/0B1uBdVuLED5NS1RFbEFuSS1iejA
> > > >
> > > > (Took them with 7-8s interval I think, hope it's ok).
> > > >
> > > > I took the thread dumps once JMeter started filling in errors, which
> is
> > > > somewhat after my CPU usage hit 100% (and music playback started to
> > > distort
> > > > etc.).
> > > >
> > > > I also created the head dump, note that I took it after running the
> > test
> > > > and after restarting JMeter, as JMeter froze during the test and I
> had
> > to
> > > > kill it. DId you mean I should take it *during* the test?
> > >
> > >
> > > YES.
> > >
> > >
> > > > Anyway, before
> > > > sharing it I want to make sure it can't contain anything
> confidential?
> > > > (parts of my test plan, passwords, environment variables,
> anything...)
> > > >
> > > > Tuukka
> > > >
> > > >
> > > >
> > > > On Wed, Feb 1, 2017 at 10:10 AM, UBIK LOAD PACK Support <
> > > > support@ubikloadpack.com> wrote:
> > > >
> > > > > Hello,
> > > > > To investigate the problem:
> > > > > 1/ to create  thread dumps, install a JDK of same version as the
> one
> > > > > running your JMeter.
> > > > >
> > > > > Run until you get the problem, then call using same user as the one
> > > > running
> > > > > JMeter 3 times at 5s interval:
> > > > >  <jdk_home>/bin/jstack -l <pid of JMeter> >> output_file.txt
> > > > >
> > > > > It will also help if you provide the structure of your Test Plan.
> > > > >
> > > > > 2/ It would also help if you can provide a Heap Dump. With JMeter
> > GUI,
> > > > use
> > > > > menu Help > Create a Heap Dump
> > > > > This will create a Heap Dump (rather big file) , it would help a
> lot
> > if
> > > > you
> > > > > can make it available.
> > > > >
> > > > > Regards
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Feb 1, 2017 at 9:03 AM, Tuukka Mustonen <
> > > > tuukka.mustonen@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > @Philippe:
> > > > > >
> > > > > > Yeah, I am normally running in non-GUI mode, but to
> > > > > > study/inspect/understand this problem I am running in GUI mode.
> > > JMeter
> > > > > 2.13
> > > > > > works just fine in GUI mode for these tests so JMeter 3.x should
> > too.
> > > > > >
> > > > > > I'll give the nightly build a try.
> > > > > >
> > > > > > Tuukka
> > > > > >
> > > > > >
> > > > > > On Wed, Feb 1, 2017 at 9:59 AM, Philippe Mouawad <
> > > > > > philippe.mouawad@gmail.com
> > > > > > > wrote:
> > > > > >
> > > > > > > On Wednesday, February 1, 2017, Tuukka Mustonen <
> > > > > > tuukka.mustonen@gmail.com
> > > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > @Philippe:
> > > > > > > >
> > > > > > > > 1. Yeah, I compared them with the same plan, same
machine.
> > Should
> > > > be
> > > > > > > > everything equal.
> > > > > > > > 2. I tried both the defaults in jmeter.bat (-Xms512m
> -Xmx512m)
> > > and
> > > > as
> > > > > > > > slightly increased (-Xms512m -Xmx1024m). I didn't
tweak them
> > much
> > > > > > > because I
> > > > > > > > don't believe the issue lies there.
> > > > > > > > 3. No, I do just simple HTTP GETs (against RESTy API).
> > > > > > > > 4. I am not familiar with that but I can try. If you
have
> > > > > instructions
> > > > > > to
> > > > > > > > point to, please do.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > 5. I am running with GUI.
> > > > > > >
> > > > > > >
> > > > > > > This is a known anti-pattern .
> > > > > > > switch to non gui mode if you want correct results.
> > > > > > > http://jmeter.apache.org/usermanual/best-practices.html
> > > > > > >
> > > > > > > http://www.ubik-ingenierie.com/blog/jmeter_performance_
> > > tuning_tips/
> > > > > > >
> > > > > > >
> > > > > > > > About file sending:
> > > > > > > >
> > > > > > > > 6. Nope. I'm just doing simple GETs.
> > > > > > >
> > > > > > >  Could you give nightly build a try ?
> > > > > > > http://jmeter.apache.org/nightly.html
> > > > > > >
> > > > > > >
> > > > > > > > Tuukka
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Feb 1, 2017 at 9:43 AM, Philippe Mouawad <
> > > > > > > > philippe.mouawad@gmail.com <javascript:;>
> > > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Also are you sending files in post body ?
> > > > > > > > >
> > > > > > > > > Thank you
> > > > > > > > >
> > > > > > > > > On Wednesday, February 1, 2017, Philippe Mouawad
<
> > > > > > > > > philippe.mouawad@gmail.com <javascript:;>>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hello,
> > > > > > > > > > Few questions, please try to answer them
all:
> > > > > > > > > >
> > > > > > > > > > 1/ Did you compare 2.13 and 3.1 with same
plan/sape
> > > machine...
> > > > > > > > everything
> > > > > > > > > > equal ?
> > > > > > > > > > 2/ What are your Xmx settings
> > > > > > > > > > 3/ does your plan download embedded resources
?
> > > > > > > > > > 4/ when issue happens can you make 3 thread
dumps at 5
> > > seconds
> > > > > > > interval
> > > > > > > > > > and send file ? (use jstack)
> > > > > > > > > > 5/ are you running gui or non gui ?
> > > > > > > > > >
> > > > > > > > > > Thanks
> > > > > > > > > > Regards
> > > > > > > > > >
> > > > > > > > > > On Wednesday, February 1, 2017, Tuukka Mustonen
<
> > > > > > > > > tuukka.mustonen@gmail.com <javascript:;>
> > > > > > > > > > <javascript:_e(%7B%7D,'cvml','tuukka.mustonen@gmail.com
> > > > > > > <javascript:;>');>>
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > >> Hi,
> > > > > > > > > >>
> > > > > > > > > >> After upgrading to JMeter 3.0 or 3.1
I experience high
> CPU
> > > > usage
> > > > > > > that
> > > > > > > > > >> renders JMeter unusable. CPU usage ramps
quickly up and
> > > after
> > > > > > having
> > > > > > > > > >> ~100-200 concurrent users or so (in
~30 seconds), it
> hits
> > > 100%
> > > > > on
> > > > > > my
> > > > > > > > > >> i7-4600. Shortly after, whole JMeter
freezes and I have
> to
> > > > kill
> > > > > > it.
> > > > > > > > > >>
> > > > > > > > > >> On 2.13 everything goes smoothly (with
default
> settings).
> > > CPU
> > > > > > usage
> > > > > > > > > stays
> > > > > > > > > >> at ~30% even with 500 users.
> > > > > > > > > >>
> > > > > > > > > >> The experience is similar on both Windows
10 and Linux
> > > (Ubuntu
> > > > > > 14.04
> > > > > > > > LTS
> > > > > > > > > >> guest in Virtualbox).
> > > > > > > > > >>
> > > > > > > > > >> The problems I saw in JMeter 3:
> > > > > > > > > >>
> > > > > > > > > >> - java.net.SocketException: Permission
denied: connect
> > > > > > > > > >> - After a while it runs out of heap
space
> > > > > > > > > >>
> > > > > > > > > >> For "Permission denied: connect" I tried
to set
> > > > > > > > > >> java.net.preferIPv4Stack=true
> > > > > > > > > >> (in a few places to make sure it would
really be in use)
> > and
> > > > > tried
> > > > > > > to
> > > > > > > > > run
> > > > > > > > > >> JMeter as administrator (in Win). No
effect.
> > > > > > > > > >>
> > > > > > > > > >> I also tried increasing heap size, but
that didn't help
> > > > either.
> > > > > > And
> > > > > > > I
> > > > > > > > > >> think
> > > > > > > > > >> memory size is not the root cause here
- rather the
> > result.
> > > > > > > > > >>
> > > > > > > > > >> I am normally connected to VPN but I
also tried
> disabling
> > > it.
> > > > No
> > > > > > > > effect.
> > > > > > > > > >>
> > > > > > > > > >> I tried disabling with and without IPv6
enabled in
> Windows
> > > > > network
> > > > > > > > > >> connection settings. No effect.
> > > > > > > > > >>
> > > > > > > > > >> I am running a few extra JMeter plugins.
Tried cutting
> my
> > > test
> > > > > > > > scenario
> > > > > > > > > to
> > > > > > > > > >> minimum. No effect.
> > > > > > > > > >>
> > > > > > > > > >> I am running anti-virus software that
I cannot disable,
> > due
> > > to
> > > > > > this
> > > > > > > > > being
> > > > > > > > > >> forced by IT department. However, it's
not a problem
> with
> > > 2.13
> > > > > so
> > > > > > I
> > > > > > > > > assume
> > > > > > > > > >> it shouldn't be a problem here either.
> > > > > > > > > >>
> > > > > > > > > >> FINALLY, once I set HTTP request implementation
from
> > Apache
> > > > > > > > HttpClient 4
> > > > > > > > > >> to
> > > > > > > > > >> HttpClient 3.1 or JAVA, the problem
vanishes - CPU stays
> > > where
> > > > > it
> > > > > > > did
> > > > > > > > > with
> > > > > > > > > >> JMeter 2.13.
> > > > > > > > > >>
> > > > > > > > > >> I'm running Oracle JVM 8 (update 121
on Win, update 111
> on
> > > > > Linux).
> > > > > > > > > >>
> > > > > > > > > >> Why does (the recommended) HttpClient
4 cause problems
> > here?
> > > > Is
> > > > > > > this a
> > > > > > > > > >> know
> > > > > > > > > >> bug/issue? JMeter 3.x is running newer
version compared
> to
> > > > 2.13,
> > > > > > but
> > > > > > > > > what
> > > > > > > > > >> change in HttpClient 4 might cause the
problems
> described?
> > > > > > > > > >>
> > > > > > > > > >> Any ideas?
> > > > > > > > > >>
> > > > > > > > > >> Regards,
> > > > > > > > > >> Tuukka
> > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Cordialement.
> > > > > > > > > > Philippe Mouawad.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Cordialement.
> > > > > > > > > Philippe Mouawad.
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Cordialement.
> > > > > > > Philippe Mouawad.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Cordialement.
> > > Philippe Mouawad.
> > >
> >
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message