jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tuukka Mustonen <tuukka.musto...@gmail.com>
Subject Re: High CPU utilization in JMeter 3.x with HttpClient 4 leads to freeze
Date Thu, 02 Feb 2017 05:50:23 GMT
@Felix: Yeah, sorry about that. I had some private exchange with Philippe
who asked my to create new proper dumps. I guess it doesn't hurt to share
them with everyone. You can find them here:

JMeter 2.13:

https://drive.google.com/drive/folders/0B1uBdVuLED5NYTktXzI1TzlIOUk?
usp=sharing

JMeter 3.1:

https://drive.google.com/drive/folders/0B1uBdVuLED5NejZaanZ2UHNlblk?
usp=sharing

Answers to questions:

- I have not configured JVM so I guess I don't have security manager enabled
- For test plan, you can check
https://bz.apache.org/bugzilla/attachment.cgi?id=34703 that Philippe
simplified from my test plan.
- Not running proxy per se, but I am running over company's VPN connection.
However, I've also tried without it, and there was no difference.
- Running JMeter locally on my machine.
- I am testing solely on Windows now, as I think Virtualboxed linux guest
adds complexity to the system. I guess I could try sniffing what Windows
says about the connections, however.

Tuukka


On Wed, Feb 1, 2017 at 11:00 PM, Felix Schumacher <
felix.schumacher@internetallee.de> wrote:

>
>
> Am 1. Februar 2017 12:57:01 MEZ schrieb Tuukka Mustonen <
> tuukka.mustonen@gmail.com>:
> >Ok I re-ran the test.
> >
> >Took Heap Dump just before JMeter froze.
> >
> >Took Thread Dumps right after JMeter froze.
> >
> >https://drive.google.com/drive/folders/0B1uBdVuLED5NNFNqallGam13MTA
>
> That folder seems to be empty now.
>
> Are you running the tests with a security manager enabled?
>
> Can you share the test or at least the used requests?
>
> Are you using a proxy and if so, how is it configured?
>
> Is jmeter running locally, or over something like rdp?
>
> Can you look at the used network connections like netstat -etapn would do
> on Linux?
>
> Regards,
>  Felix
>
> >
> >...
> >
> >For the fun of it, I also tried upgrading to the latest HttpClient
> >4.5.3
> >version. No effect.
> >
> >I also noticed that if I ping (google) during the perf testing, Windows
> >ping reports "General failure." when JMeter breaks. After killing it,
> >connectivity restores and there are replies from the server again. It
> >seems
> >like JMeter/HttpClient kills my whole network connection...
> >
> >Tuukka
> >
> >
> >On Wed, Feb 1, 2017 at 11:44 AM, Philippe Mouawad <
> >philippe.mouawad@gmail.com> wrote:
> >
> >> On Wed, Feb 1, 2017 at 9:55 AM, Tuukka Mustonen
> ><tuukka.mustonen@gmail.com
> >> >
> >> wrote:
> >>
> >> > Hi, I'm running solely on Win 10 now. In re-attempts I think I saw
> >> slightly
> >> > different behavior in my Linux guest. I'll write more about that
> >later,
> >> but
> >> > for simplicity I'm testing only on Windows host now:
> >> >
> >> > Thread dumps are downloadable at:
> >> > https://drive.google.com/drive/folders/0B1uBdVuLED5NS1RFbEFuSS1iejA
> >> >
> >> > (Took them with 7-8s interval I think, hope it's ok).
> >> >
> >> > I took the thread dumps once JMeter started filling in errors,
> >which is
> >> > somewhat after my CPU usage hit 100% (and music playback started to
> >> distort
> >> > etc.).
> >> >
> >> > I also created the head dump, note that I took it after running the
> >test
> >> > and after restarting JMeter, as JMeter froze during the test and I
> >had to
> >> > kill it. DId you mean I should take it *during* the test?
> >>
> >>
> >> YES.
> >>
> >>
> >> > Anyway, before
> >> > sharing it I want to make sure it can't contain anything
> >confidential?
> >> > (parts of my test plan, passwords, environment variables,
> >anything...)
> >> >
> >> > Tuukka
> >> >
> >> >
> >> >
> >> > On Wed, Feb 1, 2017 at 10:10 AM, UBIK LOAD PACK Support <
> >> > support@ubikloadpack.com> wrote:
> >> >
> >> > > Hello,
> >> > > To investigate the problem:
> >> > > 1/ to create  thread dumps, install a JDK of same version as the
> >one
> >> > > running your JMeter.
> >> > >
> >> > > Run until you get the problem, then call using same user as the
> >one
> >> > running
> >> > > JMeter 3 times at 5s interval:
> >> > >  <jdk_home>/bin/jstack -l <pid of JMeter> >> output_file.txt
> >> > >
> >> > > It will also help if you provide the structure of your Test Plan.
> >> > >
> >> > > 2/ It would also help if you can provide a Heap Dump. With JMeter
> >GUI,
> >> > use
> >> > > menu Help > Create a Heap Dump
> >> > > This will create a Heap Dump (rather big file) , it would help a
> >lot if
> >> > you
> >> > > can make it available.
> >> > >
> >> > > Regards
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > On Wed, Feb 1, 2017 at 9:03 AM, Tuukka Mustonen <
> >> > tuukka.mustonen@gmail.com
> >> > > >
> >> > > wrote:
> >> > >
> >> > > > @Philippe:
> >> > > >
> >> > > > Yeah, I am normally running in non-GUI mode, but to
> >> > > > study/inspect/understand this problem I am running in GUI mode.
> >> JMeter
> >> > > 2.13
> >> > > > works just fine in GUI mode for these tests so JMeter 3.x
> >should too.
> >> > > >
> >> > > > I'll give the nightly build a try.
> >> > > >
> >> > > > Tuukka
> >> > > >
> >> > > >
> >> > > > On Wed, Feb 1, 2017 at 9:59 AM, Philippe Mouawad <
> >> > > > philippe.mouawad@gmail.com
> >> > > > > wrote:
> >> > > >
> >> > > > > On Wednesday, February 1, 2017, Tuukka Mustonen <
> >> > > > tuukka.mustonen@gmail.com
> >> > > > > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > @Philippe:
> >> > > > > >
> >> > > > > > 1. Yeah, I compared them with the same plan, same machine.
> >Should
> >> > be
> >> > > > > > everything equal.
> >> > > > > > 2. I tried both the defaults in jmeter.bat (-Xms512m
> >-Xmx512m)
> >> and
> >> > as
> >> > > > > > slightly increased (-Xms512m -Xmx1024m). I didn't tweak
> >them much
> >> > > > > because I
> >> > > > > > don't believe the issue lies there.
> >> > > > > > 3. No, I do just simple HTTP GETs (against RESTy API).
> >> > > > > > 4. I am not familiar with that but I can try. If you
have
> >> > > instructions
> >> > > > to
> >> > > > > > point to, please do.
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > > 5. I am running with GUI.
> >> > > > >
> >> > > > >
> >> > > > > This is a known anti-pattern .
> >> > > > > switch to non gui mode if you want correct results.
> >> > > > > http://jmeter.apache.org/usermanual/best-practices.html
> >> > > > >
> >> > > > > http://www.ubik-ingenierie.com/blog/jmeter_performance_
> >> tuning_tips/
> >> > > > >
> >> > > > >
> >> > > > > > About file sending:
> >> > > > > >
> >> > > > > > 6. Nope. I'm just doing simple GETs.
> >> > > > >
> >> > > > >  Could you give nightly build a try ?
> >> > > > > http://jmeter.apache.org/nightly.html
> >> > > > >
> >> > > > >
> >> > > > > > Tuukka
> >> > > > > >
> >> > > > > >
> >> > > > > > On Wed, Feb 1, 2017 at 9:43 AM, Philippe Mouawad <
> >> > > > > > philippe.mouawad@gmail.com <javascript:;>
> >> > > > > > > wrote:
> >> > > > > >
> >> > > > > > > Also are you sending files in post body ?
> >> > > > > > >
> >> > > > > > > Thank you
> >> > > > > > >
> >> > > > > > > On Wednesday, February 1, 2017, Philippe Mouawad
<
> >> > > > > > > philippe.mouawad@gmail.com <javascript:;>>
> >> > > > > > > wrote:
> >> > > > > > >
> >> > > > > > > > Hello,
> >> > > > > > > > Few questions, please try to answer them
all:
> >> > > > > > > >
> >> > > > > > > > 1/ Did you compare 2.13 and 3.1 with same
plan/sape
> >> machine...
> >> > > > > > everything
> >> > > > > > > > equal ?
> >> > > > > > > > 2/ What are your Xmx settings
> >> > > > > > > > 3/ does your plan download embedded resources
?
> >> > > > > > > > 4/ when issue happens can you make 3 thread
dumps at 5
> >> seconds
> >> > > > > interval
> >> > > > > > > > and send file ? (use jstack)
> >> > > > > > > > 5/ are you running gui or non gui ?
> >> > > > > > > >
> >> > > > > > > > Thanks
> >> > > > > > > > Regards
> >> > > > > > > >
> >> > > > > > > > On Wednesday, February 1, 2017, Tuukka Mustonen
<
> >> > > > > > > tuukka.mustonen@gmail.com <javascript:;>
> >> > > > > > > > <javascript:_e(%7B%7D,'cvml','tuukka.mustonen@gmail.com
> >> > > > > <javascript:;>');>>
> >> > > > > > wrote:
> >> > > > > > > >
> >> > > > > > > >> Hi,
> >> > > > > > > >>
> >> > > > > > > >> After upgrading to JMeter 3.0 or 3.1
I experience high
> >CPU
> >> > usage
> >> > > > > that
> >> > > > > > > >> renders JMeter unusable. CPU usage ramps
quickly up
> >and
> >> after
> >> > > > having
> >> > > > > > > >> ~100-200 concurrent users or so (in ~30
seconds), it
> >hits
> >> 100%
> >> > > on
> >> > > > my
> >> > > > > > > >> i7-4600. Shortly after, whole JMeter
freezes and I
> >have to
> >> > kill
> >> > > > it.
> >> > > > > > > >>
> >> > > > > > > >> On 2.13 everything goes smoothly (with
default
> >settings).
> >> CPU
> >> > > > usage
> >> > > > > > > stays
> >> > > > > > > >> at ~30% even with 500 users.
> >> > > > > > > >>
> >> > > > > > > >> The experience is similar on both Windows
10 and Linux
> >> (Ubuntu
> >> > > > 14.04
> >> > > > > > LTS
> >> > > > > > > >> guest in Virtualbox).
> >> > > > > > > >>
> >> > > > > > > >> The problems I saw in JMeter 3:
> >> > > > > > > >>
> >> > > > > > > >> - java.net.SocketException: Permission
denied: connect
> >> > > > > > > >> - After a while it runs out of heap space
> >> > > > > > > >>
> >> > > > > > > >> For "Permission denied: connect" I tried
to set
> >> > > > > > > >> java.net.preferIPv4Stack=true
> >> > > > > > > >> (in a few places to make sure it would
really be in
> >use) and
> >> > > tried
> >> > > > > to
> >> > > > > > > run
> >> > > > > > > >> JMeter as administrator (in Win). No
effect.
> >> > > > > > > >>
> >> > > > > > > >> I also tried increasing heap size, but
that didn't
> >help
> >> > either.
> >> > > > And
> >> > > > > I
> >> > > > > > > >> think
> >> > > > > > > >> memory size is not the root cause here
- rather the
> >result.
> >> > > > > > > >>
> >> > > > > > > >> I am normally connected to VPN but I
also tried
> >disabling
> >> it.
> >> > No
> >> > > > > > effect.
> >> > > > > > > >>
> >> > > > > > > >> I tried disabling with and without IPv6
enabled in
> >Windows
> >> > > network
> >> > > > > > > >> connection settings. No effect.
> >> > > > > > > >>
> >> > > > > > > >> I am running a few extra JMeter plugins.
Tried cutting
> >my
> >> test
> >> > > > > > scenario
> >> > > > > > > to
> >> > > > > > > >> minimum. No effect.
> >> > > > > > > >>
> >> > > > > > > >> I am running anti-virus software that
I cannot
> >disable, due
> >> to
> >> > > > this
> >> > > > > > > being
> >> > > > > > > >> forced by IT department. However, it's
not a problem
> >with
> >> 2.13
> >> > > so
> >> > > > I
> >> > > > > > > assume
> >> > > > > > > >> it shouldn't be a problem here either.
> >> > > > > > > >>
> >> > > > > > > >> FINALLY, once I set HTTP request implementation
from
> >Apache
> >> > > > > > HttpClient 4
> >> > > > > > > >> to
> >> > > > > > > >> HttpClient 3.1 or JAVA, the problem vanishes
- CPU
> >stays
> >> where
> >> > > it
> >> > > > > did
> >> > > > > > > with
> >> > > > > > > >> JMeter 2.13.
> >> > > > > > > >>
> >> > > > > > > >> I'm running Oracle JVM 8 (update 121
on Win, update
> >111 on
> >> > > Linux).
> >> > > > > > > >>
> >> > > > > > > >> Why does (the recommended) HttpClient
4 cause problems
> >here?
> >> > Is
> >> > > > > this a
> >> > > > > > > >> know
> >> > > > > > > >> bug/issue? JMeter 3.x is running newer
version
> >compared to
> >> > 2.13,
> >> > > > but
> >> > > > > > > what
> >> > > > > > > >> change in HttpClient 4 might cause the
problems
> >described?
> >> > > > > > > >>
> >> > > > > > > >> Any ideas?
> >> > > > > > > >>
> >> > > > > > > >> Regards,
> >> > > > > > > >> Tuukka
> >> > > > > > > >>
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > --
> >> > > > > > > > Cordialement.
> >> > > > > > > > Philippe Mouawad.
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > > --
> >> > > > > > > Cordialement.
> >> > > > > > > Philippe Mouawad.
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > Cordialement.
> >> > > > > Philippe Mouawad.
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> Cordialement.
> >> Philippe Mouawad.
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message