Return-Path: Delivered-To: apmail-geronimo-dev-archive@www.apache.org Received: (qmail 50741 invoked from network); 3 Apr 2008 03:29:06 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 3 Apr 2008 03:29:06 -0000 Received: (qmail 68850 invoked by uid 500); 3 Apr 2008 03:29:04 -0000 Delivered-To: apmail-geronimo-dev-archive@geronimo.apache.org Received: (qmail 68788 invoked by uid 500); 3 Apr 2008 03:29:04 -0000 Mailing-List: contact dev-help@geronimo.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: List-Post: Reply-To: dev@geronimo.apache.org List-Id: Delivered-To: mailing list dev@geronimo.apache.org Received: (qmail 68777 invoked by uid 99); 3 Apr 2008 03:29:04 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Apr 2008 20:29:04 -0700 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.140] (HELO brutus.apache.org) (140.211.11.140) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Apr 2008 03:28:13 +0000 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 8129C234C0B9 for ; Wed, 2 Apr 2008 20:26:24 -0700 (PDT) Message-ID: <120325915.1207193184527.JavaMail.jira@brutus> Date: Wed, 2 Apr 2008 20:26:24 -0700 (PDT) From: "Kevan Miller (JIRA)" To: dev@geronimo.apache.org Subject: [jira] Closed: (GERONIMO-3687) classloader deadlock during server startup In-Reply-To: <14662169.1197055423237.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/GERONIMO-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevan Miller closed GERONIMO-3687. ---------------------------------- Resolution: Fixed Fix Version/s: 2.2 The best I can figure is that some JVM's are no longer using AbstractList.iterator() calls in extended for loops. From stack traces, the deadlock was occurring on the for loop. Since the static initializer (which was invoking AbstractList.iterator()), something different was obviously going on. I changed the static for loop to actually use an extended for loop, e.g.: for (ClassFileTransformer transformer : transformers) { } Pretty much a nothing-up-my-sleeve-rockie hack... but it's fixed my leopard problem. Pretty sure it will fix Jarek's problem, also. > classloader deadlock during server startup > ------------------------------------------ > > Key: GERONIMO-3687 > URL: https://issues.apache.org/jira/browse/GERONIMO-3687 > Project: Geronimo > Issue Type: Bug > Security Level: public(Regular issues) > Affects Versions: 2.0.2, 2.1 > Reporter: Kevan Miller > Assignee: Kevan Miller > Fix For: 2.0.x, 2.1.1, 2.2 > > > I've been seeing regular deadlocks running Geronimo on Leopard. > It's the same basic scenario as found in GERONIMO-3141. For some reason, the work around in 3141 doesn't work for me on Leopard. > Just another hack and I'm sure we can fix this... > Here's background info... > $ ./geronimo.sh run > Using GERONIMO_BASE: /Users/kevan/geronimo-jetty6-jee5-2.0.2 > Using GERONIMO_HOME: /Users/kevan/geronimo-jetty6-jee5-2.0.2 > Using GERONIMO_TMPDIR: var/temp > Using JRE_HOME: /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home > You can then send the java process a QUIT signal (kill -3 ) to have java dump out the thread stack traces. The deadlock occurs during a load of an Iterator class (IIRC). The JVM is not well behaved, IMO. Here's the thread stack traces that I get: > Full thread dump Java HotSpot(TM) Client VM (1.5.0_13-119 mixed mode): > "Low Memory Detector" daemon prio=5 tid=0x01009d60 nid=0x858800 runnable [0x00000000..0x00000000] > "CompilerThread0" daemon prio=9 tid=0x01009330 nid=0x857a00 waiting on condition [0x00000000..0x00000000] > "Signal Dispatcher" daemon prio=9 tid=0x01008e60 nid=0x855e00 waiting on condition [0x00000000..0x00000000] > "Finalizer" daemon prio=8 tid=0x01007d10 nid=0x81ba00 waiting for monitor entry [0xb0a05000..0xb0a05d90] > at org.apache.geronimo.transformer.TransformerCollection.transform(TransformerCollection.java:35) > at sun.instrument.TransformerManager.transform(TransformerManager.java:122) > at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:155) > at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:82) > at java.lang.ref.Finalizer.access$100(Finalizer.java:14) > at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160) > "Reference Handler" daemon prio=10 tid=0x01007910 nid=0x81a200 in Object.wait() [0xb0984000..0xb0984d90] > at java.lang.Object.wait(Native Method) > - waiting on <0x05a735f8> (a java.lang.ref.Reference$Lock) > at java.lang.Object.wait(Object.java:474) > at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116) > - locked <0x05a735f8> (a java.lang.ref.Reference$Lock) > "main" prio=5 tid=0x010018b0 nid=0xb0801000 waiting for monitor entry [0xb07ff000..0xb0800188] > at java.lang.ClassLoader.findBootstrapClass(Native Method) > at java.lang.ClassLoader.findBootstrapClass0(ClassLoader.java:946) > at java.lang.ClassLoader.loadClass(ClassLoader.java:308) > - locked <0x05a75b78> (a sun.misc.Launcher$ExtClassLoader) > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > - locked <0x05a73660> (a sun.misc.Launcher$AppClassLoader) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:280) > - locked <0x05a73660> (a sun.misc.Launcher$AppClassLoader) > at java.lang.ClassLoader.loadClass(ClassLoader.java:251) > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:374) > - locked <0x05a73660> (a sun.misc.Launcher$AppClassLoader) > "VM Thread" prio=9 tid=0x01007060 nid=0x809800 runnable > "VM Periodic Task Thread" prio=9 tid=0x0100aa00 nid=0x859c00 waiting on condition > "Exception Catcher Thread" prio=10 tid=0x01001b00 nid=0x80ae00 runnable > Found one Java-level deadlock: > ============================= > "Finalizer": > waiting to lock monitor 0x0081b070 (object 0x05a73660, a sun.misc.Launcher$AppClassLoader), > which is held by "main" > "main": > waiting to lock monitor 0x0081b094 (object 0x09584b40, a [[I), > which is held by "Finalizer" > Java stack information for the threads listed above: > =================================================== > "Finalizer": > at org.apache.geronimo.transformer.TransformerCollection.transform(TransformerCollection.java:35) > at sun.instrument.TransformerManager.transform(TransformerManager.java:122) > at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:155) > at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:82) > at java.lang.ref.Finalizer.access$100(Finalizer.java:14) > at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160) > "main": > at java.lang.ClassLoader.findBootstrapClass(Native Method) > at java.lang.ClassLoader.findBootstrapClass0(ClassLoader.java:946) > at java.lang.ClassLoader.loadClass(ClassLoader.java:308) > - locked <0x05a75b78> (a sun.misc.Launcher$ExtClassLoader) > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > - locked <0x05a73660> (a sun.misc.Launcher$AppClassLoader) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:280) > - locked <0x05a73660> (a sun.misc.Launcher$AppClassLoader) > at java.lang.ClassLoader.loadClass(ClassLoader.java:251) > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:374) > - locked <0x05a73660> (a sun.misc.Launcher$AppClassLoader) > Found 1 deadlock. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.