tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zampani, Michael" <zamp...@amazon.com>
Subject RE: Tomcat-7.0.23 startup deadlocks
Date Tue, 29 Nov 2011 00:28:13 GMT
> -----Original Message-----
> From: markt@apache.org [mailto:markt@apache.org]
> Sent: Monday, November 28, 2011 4:02 PM
> To: Tomcat Users List
> Subject: Re: Tomcat-7.0.23 startup deadlocks
>
> "Zampani, Michael" <zampani@amazon.com> wrote:
>
> >Hi,
> >
> >I'm trying to upgrade an existing webapp installation from 7.0.22 ->
> >7.0.23.
> >
> >On startup, it appears to deadlock, with the threads stuck in
> >ContainerBase.setRealm HostConfig.deployDirectories
> >ContainerBase.startInternal
> >
> >server.xml and thread dump attached.
> >Note that I do not define any <Realm>s in server.xml
> >
> >The webapp is simple and contains only static html, js, css, and
> >images.
> >Copying the web.xml from the example webapp shipped with tomcat
> seems
> >to fix it, but this worked in previous versions of Tomcat.
> >Not sure if it's relevant, but it is the ROOT webapp.
> >
> >I've attached a sample WAR file to reproduce it.  It should work fine
> >once deployed.  Remove WEB-INF/web.xml and restart Tomcat to see it
> >fail.
> >
> >Any ideas?
> >
> >-Michael
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> >For additional commands, e-mail: users-help@tomcat.apache.org
>
> Attachments are usually blocked. Better to paste in-line.
>
> Is there an actual deadlock or do things just seem to stop moving forwards?
>
> Possible causes:
> - threaded container starts (although they are still single threaded by default)
> - caching of global web.xml to improve start times
> - changes to redeployment code to correctly redeploy on context.xml
> changes
>
> If adding an empty web.xml works around this then at least there is a simple
> fix for anyone that hits this issue (whatever it is).
>
> Mark
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org


Things certainly seem to stop moving forward.
If I attach JProfiler, I see two threads waiting on the same lock, but I'm not familiar enough
with the application to know if it's a 'real' deadlock.

The webapp can be built simply by:
rm -r webapps/*
mkdir -p webapps/ROOT/WEB-INF
echo "<html><head><title>Hello world</title></head><body>Hello
World</body></html>" > webapps/ROOT/index.html

Web.xml[1] included before(copied from the example shipped with tomcat)
Thread dump attached at [2].  Hopefully my email client doesn’t' clobber it beyond recognition.
Server.xml at [3]

[1]
<?xml version="1.0" encoding="ISO-8859-1"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
                      http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
  version="3.0"
  metadata-complete="true">

  <display-name>Welcome to Tomcat</display-name>
  <description>
     Welcome to Tomcat
  </description>

</web-app>

[2]
Full thread dump Java HotSpot(TM) Server VM (20.2-b06 mixed mode):

"pool-2-thread-1" prio=10 tid=0x0a0a0800 nid=0x326a waiting for monitor entry [0x7bb94000]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at org.apache.catalina.core.ContainerBase.setRealm(ContainerBase.java:738)
        - waiting to lock <0x80aa2d70> (a org.apache.catalina.core.StandardEngine)
        at org.apache.catalina.core.StandardEngine.getRealm(StandardEngine.java:135)
        at org.apache.catalina.core.ContainerBase.getRealm(ContainerBase.java:723)
        at org.apache.catalina.core.ContainerBase.getRealm(ContainerBase.java:723)
        at org.apache.catalina.startup.ContextConfig.authenticatorConfig(ContextConfig.java:380)
        - locked <0xa32e0780> (a org.apache.catalina.startup.ContextConfig)
        at org.apache.catalina.startup.ContextConfig.configureStart(ContextConfig.java:839)
        - locked <0xa32e0780> (a org.apache.catalina.startup.ContextConfig)
        at org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:302)
        at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
        at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5148)
        - locked <0xa3265ec8> (a org.apache.catalina.core.StandardContext)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        - locked <0xa3265ec8> (a org.apache.catalina.core.StandardContext)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:866)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:842)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:615)
        at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1095)
        at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1617)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)

"pool-1-thread-1" prio=10 tid=0x7bcde000 nid=0x3269 waiting on condition [0x7bbe5000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0xa31e0918> (a java.util.concurrent.FutureTask$Sync)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
        at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:1018)
        at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:473)
        at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1412)
        at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:312)
        at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
        at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
        at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:401)
        - locked <0x80aa2ef8> (a org.apache.catalina.core.StandardHost)
        at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:346)
        - locked <0x80aa2ef8> (a org.apache.catalina.core.StandardHost)
        at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:1117)
        - locked <0x80aa2ef8> (a org.apache.catalina.core.StandardHost)
        at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:782)
        - locked <0x80aa2ef8> (a org.apache.catalina.core.StandardHost)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        - locked <0x80aa2ef8> (a org.apache.catalina.core.StandardHost)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1525)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1515)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)

"GC Daemon" daemon prio=10 tid=0x7bca1c00 nid=0x3267 in Object.wait() [0x7bd9c000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x80a3bd78> (a sun.misc.GC$LatencyLock)
        at sun.misc.GC$Daemon.run(GC.java:100)
        - locked <0x80a3bd78> (a sun.misc.GC$LatencyLock)

"Low Memory Detector" daemon prio=10 tid=0x09f83800 nid=0x3264 runnable [0x00000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" daemon prio=10 tid=0x09f82000 nid=0x3263 waiting on condition [0x00000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" daemon prio=10 tid=0x09f76800 nid=0x3262 waiting on condition [0x00000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=10 tid=0x09f75000 nid=0x3261 waiting on condition [0x00000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=10 tid=0x09f63800 nid=0x3260 in Object.wait() [0x7c37e000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x80a3c488> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
        - locked <0x80a3c488> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=10 tid=0x09f62000 nid=0x325f in Object.wait() [0x7c3cf000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x80a3bd70> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:485)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
        - locked <0x80a3bd70> (a java.lang.ref.Reference$Lock)

"main" prio=10 tid=0x09ee2400 nid=0x325b waiting on condition [0xb7367000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0xa3182228> (a java.util.concurrent.FutureTask$Sync)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
        at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:1100)
        - locked <0x80aa2d70> (a org.apache.catalina.core.StandardEngine)
        at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:291)
        - locked <0x80aa2d70> (a org.apache.catalina.core.StandardEngine)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        - locked <0x80aa2d70> (a org.apache.catalina.core.StandardEngine)
        at org.apache.catalina.core.StandardService.startInternal(StandardService.java:443)
        - locked <0x80aa2d70> (a org.apache.catalina.core.StandardEngine)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        - locked <0x80aa2150> (a org.apache.catalina.core.StandardService)
        at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:727)
        - locked <0x80aa2140> (a [Lorg.apache.catalina.Service;)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        - locked <0x80a3d1f0> (a org.apache.catalina.core.StandardServer)
        at org.apache.catalina.startup.Catalina.start(Catalina.java:624)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:322)
        at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:450)

"VM Thread" prio=10 tid=0x09f5e000 nid=0x325e runnable

"GC task thread#0 (ParallelGC)" prio=10 tid=0x09ee9800 nid=0x325c runnable

"GC task thread#1 (ParallelGC)" prio=10 tid=0x09eeb000 nid=0x325d runnable

"VM Periodic Task Thread" prio=10 tid=0x09f85800 nid=0x3265 waiting on condition

JNI global references: 1019

Heap
 PSYoungGen      total 15360K, used 11150K [0xa2f70000, 0xa4090000, 0xb4210000)
  eden space 13184K, 84% used [0xa2f70000,0xa3a538b0,0xa3c50000)
  from space 2176K, 0% used [0xa3c50000,0xa3c50000,0xa3e70000)
  to   space 2176K, 0% used [0xa3e70000,0xa3e70000,0xa4090000)
 PSOldGen        total 35136K, used 1757K [0x80a10000, 0x82c60000, 0xa2f70000)
  object space 35136K, 5% used [0x80a10000,0x80bc77a8,0x82c60000)
 PSPermGen       total 16384K, used 9687K [0x7ca10000, 0x7da10000, 0x80a10000)
  object space 16384K, 59% used [0x7ca10000,0x7d385d90,0x7da10000)


[3]
<?xml version='1.0' encoding='utf-8'?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<!-- Note:  A "Server" is not itself a "Container", so you may not
     define subcomponents such as "Valves" at this level.
     Documentation at /docs/config/server.html
 -->
<Server port="8005" shutdown="SHUTDOWN">
  <!--Initialize Jasper prior to webapps are loaded. Documentation at /docs/jasper-howto.html
-->
  <Listener className="org.apache.catalina.core.JasperListener" />
  <!-- Prevent memory leaks due to use of particular java/javax APIs-->
  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

  <!-- A "Service" is a collection of one or more "Connectors" that share
       a single "Container" Note:  A "Service" is not itself a "Container",
       so you may not define subcomponents such as "Valves" at this level.
       Documentation at /docs/config/service.html
   -->
  <Service name="Catalina">

    <!-- A "Connector" represents an endpoint by which requests are received
         and responses are returned. Documentation at :
         Java HTTP Connector: /docs/config/http.html (blocking & non-blocking)
         Java AJP  Connector: /docs/config/ajp.html
         APR (HTTP/AJP) Connector: /docs/apr.html
         Define a non-SSL HTTP/1.1 Connector on port 8080
    -->
    <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" server="Server"/>
    <!-- Define a SSL HTTP/1.1 Connector on port 8443
         This connector uses the JSSE configuration, when using APR, the
         connector should be using the OpenSSL style configuration
         described in the APR documentation -->
    <!--
    <Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
               maxThreads="150" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS" server="Server"/>
    -->


    <!-- An Engine represents the entry point (within Catalina) that processes
         every request.  The Engine implementation for Tomcat stand alone
         analyzes the HTTP headers included with the request, and passes them
         on to the appropriate Host (virtual host).
         Documentation at /docs/config/engine.html -->

    <!-- You should set jvmRoute to support load-balancing via AJP ie :
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
    -->
    <Engine name="Catalina" defaultHost="localhost">

      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">

        <!-- Access log processes all example.
             Documentation at: /docs/config/valve.html
             Note: The pattern used is equivalent to using pattern="common" -->
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log." suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />

      </Host>
    </Engine>
  </Service>
</Server>
Mime
View raw message