Return-Path: X-Original-To: apmail-jakarta-dev-archive@minotaur.apache.org Delivered-To: apmail-jakarta-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E8F479562 for ; Tue, 27 Sep 2011 09:53:02 +0000 (UTC) Received: (qmail 12639 invoked by uid 500); 27 Sep 2011 09:53:02 -0000 Delivered-To: apmail-jakarta-dev-archive@jakarta.apache.org Received: (qmail 12321 invoked by uid 500); 27 Sep 2011 09:53:01 -0000 Mailing-List: contact dev-help@jakarta.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@jakarta.apache.org Delivered-To: mailing list dev@jakarta.apache.org Received: (qmail 12310 invoked by uid 99); 27 Sep 2011 09:53:01 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Sep 2011 09:53:01 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of philippe.mouawad@gmail.com designates 209.85.160.172 as permitted sender) Received: from [209.85.160.172] (HELO mail-gy0-f172.google.com) (209.85.160.172) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Sep 2011 09:52:52 +0000 Received: by gyd12 with SMTP id 12so6234345gyd.31 for ; Tue, 27 Sep 2011 02:52:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=ev6j6DP9q4bbBCVYI3UicUoLWbUXMM5nGd2Kpd7Od84=; b=SO5BkvVqBZWcCH0xIVCawMaUkmWoUi0d/lIW+xlvFvu6iOfh+USdNayZGIyi4DE3tE e+Emu5RIILY9eNgknTOgEVuqQ2jofxsQ6IARJVAGHkabPSwSZ3rcHMFqfkBnqvoJmCx1 2R+GO+hlLd+UdqPInkYRsuKi32O+feAvgF1hk= MIME-Version: 1.0 Received: by 10.236.183.129 with SMTP id q1mr46161275yhm.35.1317117151585; Tue, 27 Sep 2011 02:52:31 -0700 (PDT) Received: by 10.236.208.228 with HTTP; Tue, 27 Sep 2011 02:52:31 -0700 (PDT) In-Reply-To: References: <4E7BBC7C.4020602@apache.org> <4E7CB0A5.5070503@apache.org> <4E7E6FCF.4000201@apache.org> <4E7E79B4.9090806@apache.org> Date: Tue, 27 Sep 2011 11:52:31 +0200 Message-ID: Subject: Re: [VOTE] Release JMeter 2.5.1 RC1 From: Philippe Mouawad To: dev@jakarta.apache.org Content-Type: multipart/alternative; boundary=20cf305b122ef2a60204ade93b79 X-Virus-Checked: Checked by ClamAV on apache.org --20cf305b122ef2a60204ade93b79 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hello St=E9phane Hoblinfre, Can you test with last build, issue you submitted has been fixed. Regards Philippe 2011/9/26 St=E9phane Hoblingre > Yes true, the bug happens also with the DummySampler of our plugins ( > http://code.google.com/p/jmeter-plugins/) and appeared from 2.5.1 only. > > Stef > > On Mon, Sep 26, 2011 at 9:18 PM, sebb wrote: > > > On 26 September 2011 09:32, Philippe Mouawad > > > wrote: > > > Hello I attached all infos on ISSUE: > > > > > > - Test case > > > - Log file in DEBUG > > > - Thread Dump > > > > > > Note that I reproduced it every time with what I described in issue. > > > > I can also now reproduce the deadlock easily with HC4 and mirror server= . > > I was using a simplified test case that had caused the deadlock prior > > to my fix, but was not causing the dealock afterwards. > > This was because it only had one sample - that had been sufficient > > originally, but was not sufficient later. > > Sorry, that's my fault; should have stayed with the original test case. > > > > The fix I made to HC4 - overriding the retryRequest() method - > > actually only works for a single sampler, because the "interrupted" > > variable is fetched from the class instance that was used to create > > the override. I introduced the "interrupted" variable, because the fix > > from https://issues.apache.org/jira/browse/HTTPCLIENT-1120 did not > > seem to work. I need to look at that again. This is needed to stop HC4 > > from retrying aborted requests, which I think is a separate issue from > > the deadlocks. > > > > I now think there's a subtle bug in JMeterThread. > > I had expected the interrupt to complete, and then the sample to > > complete, thus the shutdown code would run after the interrupt code. > > However, it looks as though the sample can complete before the > > interrupt method returns. > > This allows the thread to run the shutdown code whilst the interrupt > > is still in progress. > > > > In turn this triggers the deadlock issue in HC4, but could equally > > cause issues with other samplers. > > The code needs to ensure that shutdown cannot happen whilst an > > interrupt is in progress. > > > > I'll commit a fix to SVN ASAP. > > > > Sorry Milamber, but I think we need to cancel this RC vote. > > > > > I also attached a patch that works on my tests. > > > I let you check. > > > > > > Regards > > > Philippe Mouawad > > > > > > On Sun, Sep 25, 2011 at 1:46 PM, sebb wrote: > > > > > >> On 25 September 2011 12:22, sebb wrote: > > >> > On 25 September 2011 01:45, Milamber wrote: > > >> >> > > >> >> > > >> >> Le 25/09/2011 00:08, sebb a ecrit : > > >> >>> On 25 September 2011 01:03, Milamber wrote= : > > >> >>> > > >> >>>> > > >> >>>> Le 23/09/2011 23:38, sebb a ecrit : > > >> >>>> > > >> >>>>> On 23 September 2011 23:17, sebb wrote: > > >> >>>>> > > >> >>>>> > > >> >>>>>> On 23 September 2011 18:14, sebb wrote: > > >> >>>>>> > > >> >>>>>> > > >> >>>>>>> On 23 September 2011 17:15, Milamber > > wrote: > > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>>> Hello, > > >> >>>>>>>> > > >> >>>>>>>> It's seems all open bugs with 2.5.1RC1 are closed. > > >> >>>>>>>> I will start the RC2 process tomorrow. > > >> >>>>>>>> > > >> >>>>>>>> > > >> >>>>>>> OK. > > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>> Just found a problem with the HC4 retries - they prevent > StopTest > > >> from working. > > >> >>>>>> > > >> >>>>>> There's a deadlock in the code that prevents the interrupt fr= om > > >> >>>>>> working when there is a retry. > > >> >>>>>> [At the back of my mind, I thought there was another reason w= hy > I > > >> set > > >> >>>>>> the retry count to 0!] > > >> >>>>>> > > >> >>>>>> Shutdown works fine, because it does not try to interrupt the > > >> sample. > > >> >>>>>> > > >> >>>>>> I think there's a work-round I can use in the JMeter code, bu= t > I > > >> don't > > >> >>>>>> think we should release the code as is. > > >> >>>>>> > > >> >>>>>> Sorry. > > >> >>>>>> > > >> >>>>>> The Java and HC3.1 samplers work fine; it's only HC4 that has > the > > >> problem. > > >> >>>>>> > > >> >>>>>> I'll let you know if there's a solution ASAP. > > >> >>>>>> > > >> >>>>>> > > >> >>>>> URL: http://svn.apache.org/viewvc?rev=3D1175069&view=3Drev > > >> >>>>> Log: > > >> >>>>> Temporary hack to work round > > >> >>>>> > > >> >>>>> > > >> >>>> This temporary hack don't always work for me. When I call Stop > > command > > >> >>>> at the beginning of a test (before end of ramp up), I have the > same > > >> >>>> deadlock. > > >> >>>> (but sometimes the stop works fine.) > > >> >>>> > > >> >>> Can you do a thread dump when this happens? > > >> >>> > > >> >> > > >> >> > > >> >> Found one Java-level deadlock: > > >> >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D > > >> >> "Thread-205": > > >> >> waiting to lock monitor 0x0000000000d0bf78 (object > > 0x00000000e2c89ba8, > > >> >> a org.apache.http.impl.conn.SingleClientConnManager), > > >> >> which is held by "Thread Group 1-1" > > >> >> "Thread Group 1-1": > > >> >> waiting to lock monitor 0x000000000205e638 (object > > 0x00000000e3425510, > > >> >> a org.apache.http.impl.conn.SingleClientConnManager$ConnAdapter), > > >> >> which is held by "Thread-205" > > >> >> > > >> >> Java stack information for the threads listed above: > > >> >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > >> >> "Thread-205": > > >> >> at > > >> >> > > >> > > > org.apache.http.impl.conn.SingleClientConnManager.releaseConnection(Singl= eClientConnManager.java:258) > > >> >> - waiting to lock <0x00000000e2c89ba8> (a > > >> >> org.apache.http.impl.conn.SingleClientConnManager) > > >> >> at > > >> >> > > >> > > > org.apache.http.impl.conn.AbstractClientConnAdapter.abortConnection(Abstr= actClientConnAdapter.java:323) > > >> >> - locked <0x00000000e3425510> (a > > >> >> org.apache.http.impl.conn.SingleClientConnManager$ConnAdapter) > > >> >> at > > >> >> > > >> > > > org.apache.http.client.methods.HttpRequestBase.abort(HttpRequestBase.java= :161) > > >> >> at > > >> >> > > >> > > > org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.interrupt(HTTPHC4Impl= .java:1087) > > >> >> at > > >> >> > > >> > > > org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.interrupt(HTTPSa= mplerProxy.java:77) > > >> >> at > > >> >> > > org.apache.jmeter.threads.JMeterThread.interrupt(JMeterThread.java:580) > > >> >> at > > >> >> > > >> > > > org.apache.jmeter.engine.StandardJMeterEngine.tellThreadsToStop(StandardJ= MeterEngine.java:552) > > >> >> at > > >> >> > > >> > > > org.apache.jmeter.engine.StandardJMeterEngine.access$300(StandardJMeterEn= gine.java:58) > > >> >> at > > >> >> > > >> > > > org.apache.jmeter.engine.StandardJMeterEngine$StopTest.run(StandardJMeter= Engine.java:284) > > >> >> at java.lang.Thread.run(Thread.java:722) > > >> >> "Thread Group 1-1": > > >> >> at > > >> >> > > >> > > > org.apache.http.impl.conn.AbstractPooledConnAdapter.detach(AbstractPooled= ConnAdapter.java:106) > > >> >> - waiting to lock <0x00000000e3425510> (a > > >> >> org.apache.http.impl.conn.SingleClientConnManager$ConnAdapter) > > >> >> at > > >> >> > > >> > > > org.apache.http.impl.conn.SingleClientConnManager.shutdown(SingleClientCo= nnManager.java:342) > > >> >> - locked <0x00000000e2c89ba8> (a > > >> >> org.apache.http.impl.conn.SingleClientConnManager) > > >> >> at > > >> >> > > >> > > > org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.closeThreadLocalConne= ctions(HTTPHC4Impl.java:1072) > > >> >> at > > >> >> > > >> > > > org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.threadFinished(HTTPHC= 4Impl.java:1061) > > >> >> at > > >> >> > > >> > > > org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.threadFinished(H= TTPSamplerProxy.java:71) > > >> >> at > > >> >> > > >> > > > org.apache.jmeter.threads.JMeterThread$ThreadListenerTraverser.addNode(JM= eterThread.java:553) > > >> >> at > > >> >> > > org.apache.jorphan.collections.HashTree.traverseInto(HashTree.java:986) > > >> >> at > > >> org.apache.jorphan.collections.HashTree.traverse(HashTree.java:969) > > >> >> at > > >> >> > > >> > > > org.apache.jmeter.threads.JMeterThread.threadFinished(JMeterThread.java:5= 28) > > >> >> at > > org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:308) > > >> >> at java.lang.Thread.run(Thread.java:722) > > >> >> > > >> >> Found 1 deadlock. > > >> >> > > >> >> > > >> >> > > >> >> Full thread dump in attachment. > > >> > > > >> > Thanks! > > >> > > > >> >> Test case is the SimpleTest.jmx, stop after 6-7 VU. > > >> >> For this thread dump, I wait the end of ramp up for 100 VU, but > only > > 99 > > >> >> up. (the 'last' is the TG 1-1) > > >> >> > > >> >> > > >> >>> > > >> >>> > > >> >>>> I thinks we must release the version 2.5.1 to correct the other= s > > bugs > > >> >>>> already fixed, and add a known problem in documentation for thi= s > > >> deadlook? > > >> >>>> > > >> >>> Yes, that sounds reasonable. It's not clear yet whether this is = a > > >> >>> JMeter or HC4 problem; nor is it clear what to do to fix it. > > >> >>> Anyway, it only occurs sometimes, and it only occurs when trying > to > > >> >>> stop the test - so the GUI can just be exitted. > > >> >>> > > >> >> > > >> >> Ok, I will prepare the RC2 tomorrow. > > >> >> (Perhaps open a jmeter bugzilla for this bug?) > > >> > > > >> > I've opened https://issues.apache.org/jira/browse/HTTPCLIENT-1127i= n > > >> > case the bug is in HC4. > > >> > Looks like that may have a lock ordering problem. > > >> > > > >> > But it may be caused by a bug in the way JMeter processes shutdown= ; > > >> > I'm a bit surprised that threadFinished occurs during interrupt > > >> > processing. > > >> > I'll raise a JMeter bug and take a further look later today. > > >> > > >> Raised https://issues.apache.org/bugzilla/show_bug.cgi?id=3D51888 > > >> > > >> I don't suppose you have the corresponding JMeter log file? > > >> I think we need both dump and log to debug the problem. > > >> > > >> If not, never mind, I'll see if I can trigger the fault again. > > >> > > >> > > > >> >> Milamber > > >> >> > > >> >>> > > >> >>>> Milamber > > >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> > > >> >>>>> https://issues.apache.org/jira/browse/HTTPCLIENT-1120 > > >> >>>>> Note: copying the code from the patch did not seem to work; it > > looks > > >> >>>>> like isAborted() was not set. > > >> >>>>> Hopefully that is fixed in 4.1.3 > > >> >>>>> > > >> >>>>> That seems to have fixed it for me, or at least improved > matters. > > >> >>>>> > > >> >>>>> Still needs more testing to see if the deadlock I found - > reported > > in > > >> >>>>> https://issues.apache.org/jira/browse/HTTPCLIENT-1127 - can > still > > >> >>>>> occur. > > >> >>>>> > > >> >>>>> BTW, I found the deadlock testing against the mirror server. > > >> >>>>> > > >> >>>>> > > --------------------------------------------------------------------- > > >> >>>>> To unsubscribe, e-mail: dev-unsubscribe@jakarta.apache.org > > >> >>>>> For additional commands, e-mail: dev-help@jakarta.apache.org > > >> >>>>> > > >> >>>>> > > >> >>>>> > > >> >>>>> > > >> >>>> > > >> >>>> > > --------------------------------------------------------------------- > > >> >>>> To unsubscribe, e-mail: dev-unsubscribe@jakarta.apache.org > > >> >>>> For additional commands, e-mail: dev-help@jakarta.apache.org > > >> >>>> > > >> >>>> > > >> >>>> > > >> >>> > > --------------------------------------------------------------------- > > >> >>> To unsubscribe, e-mail: dev-unsubscribe@jakarta.apache.org > > >> >>> For additional commands, e-mail: dev-help@jakarta.apache.org > > >> >>> > > >> >>> > > >> >>> > > >> >> > > >> >> > > >> > > > >> > > >> --------------------------------------------------------------------= - > > >> To unsubscribe, e-mail: dev-unsubscribe@jakarta.apache.org > > >> For additional commands, e-mail: dev-help@jakarta.apache.org > > >> > > >> > > > > > > > > > -- > > > Cordialement. > > > Philippe Mouawad. > > > Ubik-Ing=E9nierie > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscribe@jakarta.apache.org > > For additional commands, e-mail: dev-help@jakarta.apache.org > > > > > --=20 Cordialement. Philippe Mouawad. Ubik-Ing=E9nierie --20cf305b122ef2a60204ade93b79--