jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philippe Mouawad <philippe.moua...@gmail.com>
Subject Re: JMeter Performance evolution : My 2 cents
Date Sun, 27 May 2012 21:08:36 GMT
Hello Oleg,milamber,
Thanks for your investigations.

Oleg, Regarding connectionmanager, do you think it's better to use one
connmgr for all vus ?
Today we use one per VU but we reuse httpclient instance when downloading
embedded résources in parallèl mode.
What would be the compromise ?

Thanks
Regards
Philippe



On Sunday, May 27, 2012, Oleg Kalnichevski wrote:

> On Sun, 2012-05-27 at 16:47 +0100, Milamber wrote:
> >
> >
> > Le 27/05/2012 14:57, Oleg Kalnichevski a ecrit :
> > > On Sat, 2012-05-26 at 23:14 +0100, Milamber wrote:
> > >
> > > > Le 26/05/2012 21:25, Milamber a ecrit :
> > > >
> > > > > Hello,
> > > > >
> > > > > Thanks for the wiki page.
> > > > >
> > > > > I've got the test plan, and I run some tests.
> > > > >
> > > > > Target:
> > > > > 1 VM on KVM host | 2 Gb RAM | 4 vCPU
> > > > > Debian 6.0.5 (64 bits)
> > > > > Tomcat 6.0.35 (from Apache site)
> > > > > Sun Java 1.6u32 (64bits)
> > > > > Restart Tomcat before each test
> > > > > Tomcat with default params.
> > > > >
> > > > > JMeter machine (another machine) :
> > > > > 8 Gb RAM | 2 core 2.53GHz
> > > > > Debian 6.0.5  (64 bits)
> > > > > Sun Java 1.6u32 (64bits)
> > > > > JMeter only change on JAVA options: -Xloggc:/tmp/gc_jmeterXXX.txt
> > > > >
> > > > > With the GC log, I've make some stats.
> > > > >
> > > > > With the original test plan: HttpClient 4:
> > > > > (==>4.1)
> > > > > JMeter_251    Nb GC: 202    Full GC: 17    Time GC: 13.9376210
>  Full:
> > > > > 11.3386420    Total: 25.2762630
> > > > > JMeter_26    Nb GC: 216    Full GC: 18    Time GC: 14.6083630
>  Full:
> > > > > 12.0604050    Total: 26.6687680
> > > > > JMeter_27    Nb GC: 199    Full GC: 15    Time GC: 11.7238800
>  Full:
> > > > > 9.5478390    Total: 21.2717190
> > > > >
> > > > > Stats results HC4: (==>4.1)
> > > > > JMeter 2.5.1: Requests 163917 in 604.8s =  271.0/s Avg:   148
> > > > > JMeter 2.6: Requests 163208 in 605.0s =  269.7/s Avg:   155
> > > > > JMeter 2.7RC3: 164035 in 605.1s =  271.1/s Avg:   159
> > > > >
> > > > >
> > > > With JMeter trunk and HttpClient 4.2 (core, client, mime):
> > > >
> > > > JMeter_trunk_hc42    Nb GC: 131    Full GC: 1    Time GC: 5.7903490
> > > > Full: .0285440    Total: 5.8188930
> > > >
> > > > JMeter trunk with HC 4.2 (not commit): Nb Requests 164762 in 605.2s =
> > > > 272.3/s Avg:   250
> > > >
> > > >
> > > > Results (GC counts and avg req/s) with the HC 4.2 are same with HC3.1
> > > > and Java.
> > > >
> > > > => HC 4.2 seems fixed a performance issue with JVM memory?
> > > >
> > > > => Release 2.7.1 to do with HC4.2 ? (or update HC and make a RC4?)
> > > >
> > > > Milamber
> > > >
> > > >
> > > >
> > > > > Changes HttpClient to 3.1:
> > > > > JMeter_251    Nb GC: 136    Full GC: 1    Time GC: 8.6780490
>  Full:
> > > > > .0254120    Total: 8.7034610
> > > > > JMeter_26    Nb GC: 146    Full GC: 1    Time GC: 9.0567930
>  Full:
> > > > > .0305490    Total: 9.0873420
> > > > > JMeter_27    Nb GC: 137    Full GC: 1    Time GC: 6.9212530
>  Full:
> > > > > .0282870    Total: 6.9495400
> > > > >
> > > > > Stats results HC3.1:
> > > > > JMeter 2.5.1: Requests 165103 in 605.1s =  272.9/s Avg:   233
> > > > > JMeter 2.6: Requests 165063 in 605.1s =  272.8/s Avg:   214
> > > > > JMeter 2.7RC3: Requests 164860 in 604.5s =  272.7/s Avg:   180
> > > > >
> > > > > Changes to Java http client:
> > > > > JMeter_251    Nb GC: 138    Full GC: 1    Time GC: 8.1128620
>  Full:
> > > > > .0253190    Total: 8.1381810
> > > > > JMeter_26    Nb GC: 149    Full GC: 2    Time GC: 8.4811800
>  Full:
> > > > > .6600440    Total: 9.1412240
> > > > > JMeter_27    Nb GC: 137    Full GC: 1    Time GC: 6.4912220
>  Full:
> > > > > .0283680    To>  Biggest Top-Level Dominator Classes
> >
> >     Label
> >   Number of
> >    Objects
> > Used Heap Size
> > Retained Heap
> >      Size
> > Retained Heap,
> >       %
> > org.apache.jmeter.protocol.http.control.HeaderManager
> >          6,000
> >        384,000
> >     40,142,160
> >         18.26%
> > org.apache.http.impl.conn.SingleClientConnManager
> >          1,500
> >        120,000
> >     33,289,328
> >         15.14%
> > org.apache.jmeter.threads.JMeterThread
> >          1,500
> >        216,000
> >     31,718,400
> >         14.43%
> > org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy
> >          6,004
> >        384,256
> >     30,097,176
> >         13.69%
> > java.lang.Thread
> >          1,510
> >        265,760
> >     18,165,056
> >          8.26%
> > org.apache.jmeter.assertions.ResponseAssertion
> >          6,000
> >        336,000
> >      9,504,000
> >          4.32%
> > org.apache.jmeter.threads.JMeterContext
> >          1,500
> >        120,000
> >      8,230,528
> >          3.74%
> > org.apache.jmeter.sampler.TestAction
> >          6,000
> >        336,000
> >      7,680,000
> >          3.49%
> > org.apache.jmeter.threads.ThreadGroup
> >          1,501
> >         96,064
> >      5,379,528
> >          2.45%
> > org.apache.jmeter.config.ConfigTestElement
> >          1,500
> >         84,000
> >      4,648,080
> >          2.11%
> > org.apache.jmeter.protocol.http.control.CookieManager
> >          1,501
> >        108,072
> >      4,296,984
> >          1.95%
> > java.lang.Class
> >          1,759
> >         27,840
> >      4,177,528
> >          1.90%
> > org.apache.http.conn.scheme.SchemeRegistry
> >          1,500
> >         36,000
> >      3,725,520
> >          1.69%
> > org.apache.jmeter.config.Arguments
> >          1,500
> >         84,000
> >      3,180,000
> >          1.45%
> > org.apache.jmeter.control.TransactionController
> >          1,500
> >        204,000
> >      2,832,000
> >          1.29%
> >
> >       * Total:
> >         15
> >         entries
> >         40,775
> >      2,801,992
> >    207,066,288
> >
> >
> >
> >
> >
> > With JMeter Trunk + HttpClient 4.2 :
> >
> >
> >  Biggest Top-Level Dominator Classes
> >     Label
> >   Number of
> >    Objects
> > Used Heap Size
> > Retained Heap
> >      Size
> > Retained Heap,
> >       %
> > org.apache.jmeter.protocol.http.control.HeaderManager
> >          6,000
> >        384,000
> >     41,283,936
> >         22.75%
> > org.apache.jmeter.threads.JMeterThread
> >          1,500
> >        216,000
> >     31,925,472
> >         17.59%
> > org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy
> >          6,004
> >        384,256
> >     28,151,656
> >         15.51%
> > java.lang.Thread
> >          1,504
> >        264,704
> >     15,899,168
> >          8.76%
> > org.apache.jmeter.assertions.ResponseAssertion
> >          6,000
> >        336,000
> >      9,441,672
> >          5.20%
> > org.apache.jmeter.sampler.TestAction
> >          6,000
> >        336,000
> >      7,601,096
> >          4.19%
> > org.apache.jmeter.threads.JMeterContext
> >          1,332
> >        106,560
> >      7,156,624
> >          3.94%
> > org.apache.jmeter.threads.ThreadGroup
> >          1,501
> >         96,064
> >      5,353,896
> >          2.95%
> > org.apache.jmeter.config.ConfigTestElement
> >          1,500
> >         84,000
> >      5,141,184
> >          2.83%
> > org.apache.jmeter.protocol.http.control.CookieManager
> >          1,501
> >        108,072
> >      4,582,536
> >          2.52%
> > org.apache.http.impl.conn.BasicClientConnectionManager
> >          1,332
> >         85,248
> >      4,256,488
> >          2.35%
> > java.lang.Class
> >          1,228
> >         22,200
> >      4,006,160
> >          2.21%
> > org.apache.http.conn.scheme.SchemeRegistry
> >          1,332
> >         31,968
> >      3,314,016
> >          1.83%
> > org.apache.jmeter.config.Arguments
> >          1,500
> >         84,000
> >      3,161,184
> >          1.74%
> > org.apache.jmeter.control.TransactionController
> >          1,500
> >        204,000
> >      2,737,920
> >          1.51%
> > org.apache.jmeter.control.LoopController
> >          1,500
> >        144,000
> >      2,150,592
> >          1.18%
> >
> >       * Total:
> >         16
> >         entries
> >         41,234
> >      2,887,072
> >    176,163,600
> >
> >
> >
> >
> >
> >
> > When I observed in the HC4.1/JMeter, with outgoing reference for
> > SingleClientConnManager :
> > connManager org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl
> >
> > Class Name
> > | Shallow Heap | Retained Heap
> >
> ------------------------------------------------------------------------------------------------------------------------------------------
> > class org.apache.http.impl.conn.SingleClientConnManager @ 0xd8bd20a0
> > |            8 |           336
> > |- <class> org.apache.http.impl.conn.SingleClientConnManager @
> > 0xe6be9450                                  |           80 |
> > 27,008
> > |  |- connManager org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl
> > $4 @ 0xe6be9320                       |          168 |         9,536
> > |  |  |- value java.util.HashMap$Entry @ 0xe6be9258
> > |           48 |         9,784
> > |  |  |  '- [10] java.util.HashMap$Entry[16] @ 0xe6be9208
> > |          152 |         9,936
> > |  |  |     '- table java.util.HashMap @ 0xe6be91d8
> > |           64 |        10,000
> > |  |  |        '- value java.lang.ThreadLocal$ThreadLocalMap$Entry @
> > 0xe6be91b8                            |           56 |        10,056
> > |  |  |           '- [4] java.lang.ThreadLocal$ThreadLocalMap
> > $Entry[16] @ 0xe3fed728                       |          152 |
> > 11,064
> > |  |  |              '- table java.lang.ThreadLocal$ThreadLocalMap @
> > 0xe3fed710                            |           32 |        11,096
> > |  |  |                 '- threadLocals java.lang.Thread @ 0xe3fe4b00
> > Thread Group 1-1483 Thread          |          176 |        11,592
> > |  |  |                    |- [1486] java.lang.Thread[2048] @
> > 0xe3218f40                                   |       16,408 |
> > 16,408
> > |  |  |                    |  '- threads java.lang.ThreadGroup @
> > 0xe0000030  main                          |           72 |
> > 16,552
> > |  |  |                    |- <Java Local>, me java.lang.Thread @
> > 0xe3fe4b00  Thread Group 1-1483 Thread   |          176 |
> > 11,592
> > |  |  |                    |  |- [1486] java.lang.Thread[2048] @
> > 0xe3218f40                                |       16,408 |
> > 16,408
> > |  |  |                    |  |- <Java Local>, me java.lang.Thread @
> > 0xe3fe4b00  Thread Group 1-1483 Thread|          176 |        11,592
> > |  |  |                    |  |- value
> > java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xe4417ba0       |
> > 48 |            48
> > |  |  |                    |  '- Total: 3 entries
> > |     The new connection managers are definitely better in terms of code
> quality. However, I did not expect them to be any faster, though, as
> they pretty much shared the same design with the old ones. At the same
> time it does appear that BasicClientConnectionManager tends to have a
> much smaller footprint judging by the heap dumps, which is a pleasant
> surprise.
>
> What is not quite clear to me is why JMeter creates so many connection
> managers at all. Does it create a separate connection manager per
> virtual user or per request? In any case this explains why HttpJava
> tends to be faster in your tests. HttpJava uses one global connection
> pool for all HttpUrlConnection instances. Therefore it uses fewer
> physical connections to execute the same number of requests which
> usually is always faster. This would be equivalent to HttpClient using
> one pooling connection manager for all virtual users and requests.
>
> Hope this helps interpret the numbers.
>
> Oleg
>
> > The TestPlan from the wiki:
> >
> http://wiki.apache.org/jmeter/JMeterPerformance?action=AttachFile&do=view&target=TestPlan.jmx
> >
> > Run from JMeter 2.7RC3:
> > http://people.apache.org/~milamber/jmeter-2.7RC3/dist/
> >
> > Run from JMeter trung with HC4.2 (temporary)
> > http://people.apache.org/~milamber/temp/
> >
> > Milamber
> >
> >
> >
> > [1]
> > http://www.apache.org/dist/httpcomponents/httpcore/RELEASE_NOTES.txt
> > [2]
> > http://www.apache.org/dist/httpcomponents/httpclient/RELEASE_NOTES.txt
> >
> >
> >
> >
> >
> > > Oleg
> > >
> > >
> > >
> > > > > Le 26/05/2012 02:00, Philippe Mouawad a ecrit :
> > > > >
> > > > >
> > > > > > Hello,
> > > > > > I finally managed to insert images and Script:
> > > > > > http://wiki.apache.org/jmeter/JMeterPerformance
> > > > > >
> > > > > > Regards
> > > > > > Philippe
> > > > > > On Fri, May 25, 2012 at 1:48 PM, Philippe Mouawad <
> > > > > > philippe.mouawad@gmail.com> wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > > Hello,
> > > > > > > And are you able to insert images ? or Test Plan ?
> > > > > > >
> > > > > > > Sebb, I think I tried with Attachment to insert Plan or
Image,
> it proposes
> > > > > > > a link type with a protocol, but I don't see how to upload
the
> image or
> > > > > > > Test Plan.
> > > > > > >
> > > > > > > There is a button Insert/Edit image but it is disabled.
> > > > > > >
> > > > > > > Regards
> > > > > > > Philippe
> > > > > > >
> > > > > > >
> > > > > > > On Fri, May 25, 2012 at 12:06 PM, Milamber <
> milamber@apache.org> wrote:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > On



-- 
Cordialement.
Philippe Mouawad.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message