Return-Path: X-Original-To: apmail-jmeter-dev-archive@minotaur.apache.org Delivered-To: apmail-jmeter-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 27536C543 for ; Sun, 27 May 2012 21:09:07 +0000 (UTC) Received: (qmail 36666 invoked by uid 500); 27 May 2012 21:09:07 -0000 Delivered-To: apmail-jmeter-dev-archive@jmeter.apache.org Received: (qmail 36647 invoked by uid 500); 27 May 2012 21:09:07 -0000 Mailing-List: contact dev-help@jmeter.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@jmeter.apache.org Delivered-To: mailing list dev@jmeter.apache.org Received: (qmail 36639 invoked by uid 99); 27 May 2012 21:09:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 27 May 2012 21:09:07 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of philippe.mouawad@gmail.com designates 209.85.214.171 as permitted sender) Received: from [209.85.214.171] (HELO mail-ob0-f171.google.com) (209.85.214.171) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 27 May 2012 21:08:58 +0000 Received: by obfk16 with SMTP id k16so5762079obf.2 for ; Sun, 27 May 2012 14:08:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=aa8jswv06LATS0ZNXdaLm9seNOMxuIfQn4EyGaadNkQ=; b=hdB1H36yAj9/j5pLlbFyV+Woa8GiO6vMNG3Z+zaimMQ2+ayU3Lg5f6l0YTkNT3vhJl hEbUgtr02stYBNDBBLsuo8OnL1yGKlpkzhVtwjSjYejI1ZPuj3ontin6sb83yl9qaxfh J2iTZavjz5iUM+E9BW8qTossEuRD0YRDRolZr/Tz4zT9ZpcCUXgIq93aTYdtyO3MpB9Q kEDvvKHRInOfbtt/0Vy9n+jWzNAqNxriJ6W0IljJu4ihlYXXJiLW54z5ZXkLylwo1p6z wN2/zlryAAcr3Ezjw2iJjtA2FvqEcswhtkH4GvNDu7CbhyJCLJd7VQO9fbG4+cF2SBP3 hc9Q== MIME-Version: 1.0 Received: by 10.50.140.99 with SMTP id rf3mr3094081igb.25.1338152916696; Sun, 27 May 2012 14:08:36 -0700 (PDT) Received: by 10.231.102.138 with HTTP; Sun, 27 May 2012 14:08:36 -0700 (PDT) In-Reply-To: <1338142319.3487.58.camel@ubuntu> References: <4FBAB8FE.9020800@apache.org> <4FC13C26.7040007@apache.org> <4FC155AE.7080102@apache.org> <1338127040.3487.38.camel@ubuntu> <4FC24C78.5070206@apache.org> <1338142319.3487.58.camel@ubuntu> Date: Sun, 27 May 2012 23:08:36 +0200 Message-ID: Subject: Re: JMeter Performance evolution : My 2 cents From: Philippe Mouawad To: "dev@jmeter.apache.org" Content-Type: multipart/alternative; boundary=e89a8f642ec2413f3704c10b0153 --e89a8f642ec2413f3704c10b0153 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hello Oleg,milamber, Thanks for your investigations. Oleg, Regarding connectionmanager, do you think it's better to use one connmgr for all vus ? Today we use one per VU but we reuse httpclient instance when downloading embedded r=E9sources in parall=E8l mode. What would be the compromise ? Thanks Regards Philippe On Sunday, May 27, 2012, Oleg Kalnichevski wrote: > On Sun, 2012-05-27 at 16:47 +0100, Milamber wrote: > > > > > > Le 27/05/2012 14:57, Oleg Kalnichevski a ecrit : > > > On Sat, 2012-05-26 at 23:14 +0100, Milamber wrote: > > > > > > > Le 26/05/2012 21:25, Milamber a ecrit : > > > > > > > > > Hello, > > > > > > > > > > Thanks for the wiki page. > > > > > > > > > > I've got the test plan, and I run some tests. > > > > > > > > > > Target: > > > > > 1 VM on KVM host | 2 Gb RAM | 4 vCPU > > > > > Debian 6.0.5 (64 bits) > > > > > Tomcat 6.0.35 (from Apache site) > > > > > Sun Java 1.6u32 (64bits) > > > > > Restart Tomcat before each test > > > > > Tomcat with default params. > > > > > > > > > > JMeter machine (another machine) : > > > > > 8 Gb RAM | 2 core 2.53GHz > > > > > Debian 6.0.5 (64 bits) > > > > > Sun Java 1.6u32 (64bits) > > > > > JMeter only change on JAVA options: -Xloggc:/tmp/gc_jmeterXXX.txt > > > > > > > > > > With the GC log, I've make some stats. > > > > > > > > > > With the original test plan: HttpClient 4: > > > > > (=3D=3D>4.1) > > > > > JMeter_251 Nb GC: 202 Full GC: 17 Time GC: 13.9376210 > Full: > > > > > 11.3386420 Total: 25.2762630 > > > > > JMeter_26 Nb GC: 216 Full GC: 18 Time GC: 14.6083630 > Full: > > > > > 12.0604050 Total: 26.6687680 > > > > > JMeter_27 Nb GC: 199 Full GC: 15 Time GC: 11.7238800 > Full: > > > > > 9.5478390 Total: 21.2717190 > > > > > > > > > > Stats results HC4: (=3D=3D>4.1) > > > > > JMeter 2.5.1: Requests 163917 in 604.8s =3D 271.0/s Avg: 148 > > > > > JMeter 2.6: Requests 163208 in 605.0s =3D 269.7/s Avg: 155 > > > > > JMeter 2.7RC3: 164035 in 605.1s =3D 271.1/s Avg: 159 > > > > > > > > > > > > > > With JMeter trunk and HttpClient 4.2 (core, client, mime): > > > > > > > > JMeter_trunk_hc42 Nb GC: 131 Full GC: 1 Time GC: 5.7903490 > > > > Full: .0285440 Total: 5.8188930 > > > > > > > > JMeter trunk with HC 4.2 (not commit): Nb Requests 164762 in 605.2s= =3D > > > > 272.3/s Avg: 250 > > > > > > > > > > > > Results (GC counts and avg req/s) with the HC 4.2 are same with HC3= .1 > > > > and Java. > > > > > > > > =3D> HC 4.2 seems fixed a performance issue with JVM memory? > > > > > > > > =3D> Release 2.7.1 to do with HC4.2 ? (or update HC and make a RC4?= ) > > > > > > > > Milamber > > > > > > > > > > > > > > > > > Changes HttpClient to 3.1: > > > > > JMeter_251 Nb GC: 136 Full GC: 1 Time GC: 8.6780490 > Full: > > > > > .0254120 Total: 8.7034610 > > > > > JMeter_26 Nb GC: 146 Full GC: 1 Time GC: 9.0567930 > Full: > > > > > .0305490 Total: 9.0873420 > > > > > JMeter_27 Nb GC: 137 Full GC: 1 Time GC: 6.9212530 > Full: > > > > > .0282870 Total: 6.9495400 > > > > > > > > > > Stats results HC3.1: > > > > > JMeter 2.5.1: Requests 165103 in 605.1s =3D 272.9/s Avg: 233 > > > > > JMeter 2.6: Requests 165063 in 605.1s =3D 272.8/s Avg: 214 > > > > > JMeter 2.7RC3: Requests 164860 in 604.5s =3D 272.7/s Avg: 180 > > > > > > > > > > Changes to Java http client: > > > > > JMeter_251 Nb GC: 138 Full GC: 1 Time GC: 8.1128620 > Full: > > > > > .0253190 Total: 8.1381810 > > > > > JMeter_26 Nb GC: 149 Full GC: 2 Time GC: 8.4811800 > Full: > > > > > .6600440 Total: 9.1412240 > > > > > JMeter_27 Nb GC: 137 Full GC: 1 Time GC: 6.4912220 > Full: > > > > > .0283680 To> Biggest Top-Level Dominator Classes > > > > Label > > Number of > > Objects > > Used Heap Size > > Retained Heap > > Size > > Retained Heap, > > % > > org.apache.jmeter.protocol.http.control.HeaderManager > > 6,000 > > 384,000 > > 40,142,160 > > 18.26% > > org.apache.http.impl.conn.SingleClientConnManager > > 1,500 > > 120,000 > > 33,289,328 > > 15.14% > > org.apache.jmeter.threads.JMeterThread > > 1,500 > > 216,000 > > 31,718,400 > > 14.43% > > org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy > > 6,004 > > 384,256 > > 30,097,176 > > 13.69% > > java.lang.Thread > > 1,510 > > 265,760 > > 18,165,056 > > 8.26% > > org.apache.jmeter.assertions.ResponseAssertion > > 6,000 > > 336,000 > > 9,504,000 > > 4.32% > > org.apache.jmeter.threads.JMeterContext > > 1,500 > > 120,000 > > 8,230,528 > > 3.74% > > org.apache.jmeter.sampler.TestAction > > 6,000 > > 336,000 > > 7,680,000 > > 3.49% > > org.apache.jmeter.threads.ThreadGroup > > 1,501 > > 96,064 > > 5,379,528 > > 2.45% > > org.apache.jmeter.config.ConfigTestElement > > 1,500 > > 84,000 > > 4,648,080 > > 2.11% > > org.apache.jmeter.protocol.http.control.CookieManager > > 1,501 > > 108,072 > > 4,296,984 > > 1.95% > > java.lang.Class > > 1,759 > > 27,840 > > 4,177,528 > > 1.90% > > org.apache.http.conn.scheme.SchemeRegistry > > 1,500 > > 36,000 > > 3,725,520 > > 1.69% > > org.apache.jmeter.config.Arguments > > 1,500 > > 84,000 > > 3,180,000 > > 1.45% > > org.apache.jmeter.control.TransactionController > > 1,500 > > 204,000 > > 2,832,000 > > 1.29% > > > > * Total: > > 15 > > entries > > 40,775 > > 2,801,992 > > 207,066,288 > > > > > > > > > > > > With JMeter Trunk + HttpClient 4.2 : > > > > > > Biggest Top-Level Dominator Classes > > Label > > Number of > > Objects > > Used Heap Size > > Retained Heap > > Size > > Retained Heap, > > % > > org.apache.jmeter.protocol.http.control.HeaderManager > > 6,000 > > 384,000 > > 41,283,936 > > 22.75% > > org.apache.jmeter.threads.JMeterThread > > 1,500 > > 216,000 > > 31,925,472 > > 17.59% > > org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy > > 6,004 > > 384,256 > > 28,151,656 > > 15.51% > > java.lang.Thread > > 1,504 > > 264,704 > > 15,899,168 > > 8.76% > > org.apache.jmeter.assertions.ResponseAssertion > > 6,000 > > 336,000 > > 9,441,672 > > 5.20% > > org.apache.jmeter.sampler.TestAction > > 6,000 > > 336,000 > > 7,601,096 > > 4.19% > > org.apache.jmeter.threads.JMeterContext > > 1,332 > > 106,560 > > 7,156,624 > > 3.94% > > org.apache.jmeter.threads.ThreadGroup > > 1,501 > > 96,064 > > 5,353,896 > > 2.95% > > org.apache.jmeter.config.ConfigTestElement > > 1,500 > > 84,000 > > 5,141,184 > > 2.83% > > org.apache.jmeter.protocol.http.control.CookieManager > > 1,501 > > 108,072 > > 4,582,536 > > 2.52% > > org.apache.http.impl.conn.BasicClientConnectionManager > > 1,332 > > 85,248 > > 4,256,488 > > 2.35% > > java.lang.Class > > 1,228 > > 22,200 > > 4,006,160 > > 2.21% > > org.apache.http.conn.scheme.SchemeRegistry > > 1,332 > > 31,968 > > 3,314,016 > > 1.83% > > org.apache.jmeter.config.Arguments > > 1,500 > > 84,000 > > 3,161,184 > > 1.74% > > org.apache.jmeter.control.TransactionController > > 1,500 > > 204,000 > > 2,737,920 > > 1.51% > > org.apache.jmeter.control.LoopController > > 1,500 > > 144,000 > > 2,150,592 > > 1.18% > > > > * Total: > > 16 > > entries > > 41,234 > > 2,887,072 > > 176,163,600 > > > > > > > > > > > > > > When I observed in the HC4.1/JMeter, with outgoing reference for > > SingleClientConnManager : > > connManager org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl > > > > Class Name > > | Shallow Heap | Retained Heap > > > -------------------------------------------------------------------------= ----------------------------------------------------------------- > > class org.apache.http.impl.conn.SingleClientConnManager @ 0xd8bd20a0 > > | 8 | 336 > > |- org.apache.http.impl.conn.SingleClientConnManager @ > > 0xe6be9450 | 80 | > > 27,008 > > | |- connManager org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl > > $4 @ 0xe6be9320 | 168 | 9,536 > > | | |- value java.util.HashMap$Entry @ 0xe6be9258 > > | 48 | 9,784 > > | | | '- [10] java.util.HashMap$Entry[16] @ 0xe6be9208 > > | 152 | 9,936 > > | | | '- table java.util.HashMap @ 0xe6be91d8 > > | 64 | 10,000 > > | | | '- value java.lang.ThreadLocal$ThreadLocalMap$Entry @ > > 0xe6be91b8 | 56 | 10,056 > > | | | '- [4] java.lang.ThreadLocal$ThreadLocalMap > > $Entry[16] @ 0xe3fed728 | 152 | > > 11,064 > > | | | '- table java.lang.ThreadLocal$ThreadLocalMap @ > > 0xe3fed710 | 32 | 11,096 > > | | | '- threadLocals java.lang.Thread @ 0xe3fe4b00 > > Thread Group 1-1483 Thread | 176 | 11,592 > > | | | |- [1486] java.lang.Thread[2048] @ > > 0xe3218f40 | 16,408 | > > 16,408 > > | | | | '- threads java.lang.ThreadGroup @ > > 0xe0000030 main | 72 | > > 16,552 > > | | | |- , me java.lang.Thread @ > > 0xe3fe4b00 Thread Group 1-1483 Thread | 176 | > > 11,592 > > | | | | |- [1486] java.lang.Thread[2048] @ > > 0xe3218f40 | 16,408 | > > 16,408 > > | | | | |- , me java.lang.Thread @ > > 0xe3fe4b00 Thread Group 1-1483 Thread| 176 | 11,592 > > | | | | |- value > > java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xe4417ba0 | > > 48 | 48 > > | | | | '- Total: 3 entries > > | The new connection managers are definitely better in terms of cod= e > quality. However, I did not expect them to be any faster, though, as > they pretty much shared the same design with the old ones. At the same > time it does appear that BasicClientConnectionManager tends to have a > much smaller footprint judging by the heap dumps, which is a pleasant > surprise. > > What is not quite clear to me is why JMeter creates so many connection > managers at all. Does it create a separate connection manager per > virtual user or per request? In any case this explains why HttpJava > tends to be faster in your tests. HttpJava uses one global connection > pool for all HttpUrlConnection instances. Therefore it uses fewer > physical connections to execute the same number of requests which > usually is always faster. This would be equivalent to HttpClient using > one pooling connection manager for all virtual users and requests. > > Hope this helps interpret the numbers. > > Oleg > > > The TestPlan from the wiki: > > > http://wiki.apache.org/jmeter/JMeterPerformance?action=3DAttachFile&do=3D= view&target=3DTestPlan.jmx > > > > Run from JMeter 2.7RC3: > > http://people.apache.org/~milamber/jmeter-2.7RC3/dist/ > > > > Run from JMeter trung with HC4.2 (temporary) > > http://people.apache.org/~milamber/temp/ > > > > Milamber > > > > > > > > [1] > > http://www.apache.org/dist/httpcomponents/httpcore/RELEASE_NOTES.txt > > [2] > > http://www.apache.org/dist/httpcomponents/httpclient/RELEASE_NOTES.txt > > > > > > > > > > > > > Oleg > > > > > > > > > > > > > > Le 26/05/2012 02:00, Philippe Mouawad a ecrit : > > > > > > > > > > > > > > > > Hello, > > > > > > I finally managed to insert images and Script: > > > > > > http://wiki.apache.org/jmeter/JMeterPerformance > > > > > > > > > > > > Regards > > > > > > Philippe > > > > > > On Fri, May 25, 2012 at 1:48 PM, Philippe Mouawad < > > > > > > philippe.mouawad@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hello, > > > > > > > And are you able to insert images ? or Test Plan ? > > > > > > > > > > > > > > Sebb, I think I tried with Attachment to insert Plan or Image= , > it proposes > > > > > > > a link type with a protocol, but I don't see how to upload th= e > image or > > > > > > > Test Plan. > > > > > > > > > > > > > > There is a button Insert/Edit image but it is disabled. > > > > > > > > > > > > > > Regards > > > > > > > Philippe > > > > > > > > > > > > > > > > > > > > > On Fri, May 25, 2012 at 12:06 PM, Milamber < > milamber@apache.org> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On --=20 Cordialement. Philippe Mouawad. --e89a8f642ec2413f3704c10b0153--