geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevan Miller <kevan.mil...@gmail.com>
Subject Re: ActiveMQ 3.2.4-SNAPSHOT bug causes Geronimo 1.1 shutdown to hang.
Date Tue, 04 Apr 2006 14:26:41 GMT

On Apr 4, 2006, at 10:07 AM, John Sisson wrote:

> Kevin, I tried:
>
>    export GERONIMO_OPTS=-Dactivemq.broker.disable-clean-shutdown=true
>    ./geronimo.sh run --long
>
> The problem still occurred.  I'll investigate further.
>
> AFAIK, the EOFException warning message just needs to be changed to  
> a trace level message call.  Hiram can you confirm?

Juhn,
Thanks.

It's debatable rather or not the warning message should be a trace  
level message. I'll leave that for the AMQ guys...

Both sets of problems indicate that the AMQ Broker is being stopped  
prior to the stopping of the AMQ clients in the Geronimo server.

There's a third potential explanation -- the order of shutdown  
processing within Geronimo has been changed. The kernel is stopping  
the broker prior to stopping the clients...

--kevan

>
> John
>
> Kevan Miller wrote:
>> John,
>> We seem to have lost a work-around that avoided a number of  
>> ActiveMQ shutdown problems (including the one you're butting your  
>> head against). FYI, I've also noticed the following on a ctrl-c of  
>> a 1.1 server:
>>
>> 12:46:57,193 WARN  [TcpTransportChannel] Peer closed connection
>> java.io.EOFException
>>     at java.io.DataInputStream.readByte(DataInputStream.java:333)
>>     at org.activemq.io.AbstractWireFormat.readPacket 
>> (AbstractWireFormat.java:230)
>>     at org.activemq.transport.tcp.TcpTransportChannel.run 
>> (TcpTransportChannel.java:313)
>>     at java.lang.Thread.run(Thread.java:552)
>>
>> We had a work-around for these problems in 1.0 by disabling the  
>> ActiveMQ shutdown hooks, with the following in the ActiveMQ broker  
>> config plan:
>>
>>     <gbean name="SystemProperties"  
>> class="org.apache.geronimo.system.properties.SystemProperties">
>>         <attribute name="systemProperties">
>>             activemq.broker.disable-clean-shutdown=true
>>         </attribute>
>>     </gbean>
>>
>> The SystemProperties GBean is still in the ActiveMQ plan in 1.1.  
>> So, either ActiveMQ no longer honors the setting or the  
>> SystemProperties GBean isn't working. I'd guess that the GBean is  
>> broken, but probably best to test ActiveMQ using '- 
>> Dactivemq.broker.disable-clean-shutdown=true'. Could you give it a  
>> whirl?
>>
>> FYI, the deadlock condition is already reported as a problem:
>>
>> http://issues.apache.org/jira/browse/GERONIMO-1493
>>
>> Hiram created the following jira to see if the problem has been  
>> fixed on AMQ v4 rc2
>>
>> http://jira.activemq.org/jira/browse/AMQ-651
>>
>> --kevan
>>
>> On Apr 3, 2006, at 9:21 PM, John Sisson wrote:
>>
>>> I have reproduced this twice in a row on Solaris x86.
>>> Can someone with access please create a version record in JIRA  
>>> for ActiveMQ 3.2.4 so we can raise issues against it.
>>>
>>> See stack dump below:
>>>
>>> $./geronimo.sh run --long
>>> Using GERONIMO_BASE:   /home/sissonj/OpenSourceJava/asf/geronimo/ 
>>> branches/1.1/assemblies/j2ee-tomcat-server/target/geronimo-1.1- 
>>> SNAPSHOT
>>> Using GERONIMO_HOME:   /home/sissonj/OpenSourceJava/asf/geronimo/ 
>>> branches/1.1/assemblies/j2ee-tomcat-server/target/geronimo-1.1- 
>>> SNAPSHOT
>>> Using GERONIMO_TMPDIR: /home/sissonj/OpenSourceJava/asf/geronimo/ 
>>> branches/1.1/assemblies/j2ee-tomcat-server/target/geronimo-1.1- 
>>> SNAPSHOT/var/temp
>>> Using JRE_HOME:        /usr/j2se
>>> Booting Geronimo Kernel (in Java 1.4.2_10)...
>>> Configuration geronimo/rmi-naming/1.1-SNAPSHOT/car started in   
>>> 1/22   0s
>>> Configuration geronimo/j2ee-server/1.1-SNAPSHOT/car started in   
>>> 2/22   1s
>>> Configuration geronimo/j2ee-security/1.1-SNAPSHOT/car started in   
>>> 3/22   1s
>>> Configuration geronimo/system-database/1.1-SNAPSHOT/car started  
>>> in  4/22   3s
>>> Configuration geronimo/activemq-broker/1.1-SNAPSHOT/car started  
>>> in  5/22   1s
>>> Configuration geronimo/activemq/1.1-SNAPSHOT/car started in   
>>> 6/22   0s
>>> Configuration geronimo/directory/1.1-SNAPSHOT/car started in   
>>> 7/22   1s
>>> Configuration geronimo/ldap-realm/1.1-SNAPSHOT/car started in   
>>> 8/22   0s
>>> Configuration geronimo/tomcat/1.1-SNAPSHOT/car started in  9/22   3s
>>> Configuration geronimo/geronimo-gbean-deployer/1.1-SNAPSHOT/car  
>>> started in 10/22   1s
>>> Configuration geronimo/j2ee-deployer/1.1-SNAPSHOT/car started in  
>>> 11/22   1s
>>> Configuration geronimo/tomcat-deployer/1.1-SNAPSHOT/car started  
>>> in 12/22   0s
>>> Configuration geronimo/welcome-tomcat/1.1-SNAPSHOT/car started in  
>>> 13/22   0s
>>> Configuration geronimo/ldap-demo-tomcat/1.1-SNAPSHOT/car started  
>>> in 14/22   0s
>>> Configuration geronimo/servlets-examples-tomcat/1.1-SNAPSHOT/car  
>>> started in 15/22   0s
>>> Configuration geronimo/jsp-examples-tomcat/1.1-SNAPSHOT/car  
>>> started in 16/22   1s
>>> Configuration geronimo/webconsole-tomcat/1.1-SNAPSHOT/car started  
>>> in 17/22   1s
>>> Configuration geronimo/uddi-tomcat/1.1-SNAPSHOT/car started in  
>>> 18/22   0s
>>> Configuration geronimo/jmxdebug-tomcat/1.1-SNAPSHOT/car started  
>>> in 19/22   0s
>>> Configuration geronimo/daytrader-derby-tomcat/1.1-SNAPSHOT/car  
>>> started in 20/22   3s
>>> Configuration geronimo/remote-deploy-tomcat/1.1-SNAPSHOT/car  
>>> started in 21/22   0s
>>> Configuration geronimo/hot-deployer/1.1-SNAPSHOT/ 
>>> carjava.lang.IllegalArgumentException: Unable to look up  
>>> connectors for Tomcat container 'geronimo/tomcat/1.1-SNAPSHOT/car? 
>>> ServiceModule=geronimo/tomcat/1.1-SNAPSHOT/ 
>>> car,j2eeType=GBean,name=TomcatWebContainer':  
>>> java.lang.NullPointerException
>>>        at  
>>> org.apache.geronimo.tomcat.TomcatManagerImpl.getConnectorsForContain 
>>> er(TomcatManagerImpl.java:264)
>>>        at org.apache.geronimo.tomcat.TomcatManagerImpl$ 
>>> $FastClassByCGLIB$$6ff876df.invoke(<generated>)
>>>        at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53)
>>>        at  
>>> org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke 
>>> (FastMethodInvoker.java:38)
>>>        at org.apache.geronimo.gbean.runtime.GBeanOperation.invoke 
>>> (GBeanOperation.java:122)
>>>        at org.apache.geronimo.gbean.runtime.GBeanInstance.invoke 
>>> (GBeanInstance.java:848)
>>>        at org.apache.geronimo.kernel.basic.BasicKernel.invoke 
>>> (BasicKernel.java:244)
>>>        at  
>>> org.apache.geronimo.deployment.plugin.local.CommandSupport.mapContai 
>>> nersToURLs(CommandSupport.java:371)
>>>        at  
>>> org.apache.geronimo.deployment.plugin.local.CommandSupport.addWebURL 
>>> s(CommandSupport.java:294)
>>>        at  
>>> org.apache.geronimo.deployment.plugin.jmx.JMXDeploymentManager.getMo 
>>> dules(JMXDeploymentManager.java:164)
>>>        at  
>>> org.apache.geronimo.deployment.plugin.jmx.JMXDeploymentManager.getAv 
>>> ailableModules(JMXDeploymentManager.java:110)
>>>        at  
>>> org.apache.geronimo.deployment.hot.DirectoryHotDeployer.doStart 
>>> (DirectoryHotDeployer.java:141)
>>>        at  
>>> org.apache.geronimo.gbean.runtime.GBeanInstance.createInstance 
>>> (GBeanInstance.java:977)
>>>        at  
>>> org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStar 
>>> t(GBeanInstanceState.java:267)
>>>        at  
>>> org.apache.geronimo.gbean.runtime.GBeanInstanceState.start 
>>> (GBeanInstanceState.java:102)
>>>        at  
>>> org.apache.geronimo.gbean.runtime.GBeanInstanceState.startRecursive( 
>>> GBeanInstanceState.java:124)
>>>        at  
>>> org.apache.geronimo.gbean.runtime.GBeanInstance.startRecursive 
>>> (GBeanInstance.java:546)
>>>        at  
>>> org.apache.geronimo.kernel.basic.BasicKernel.startRecursiveGBean 
>>> (BasicKernel.java:391)
>>>        at  
>>> org.apache.geronimo.kernel.config.ConfigurationUtil.startConfigurati 
>>> onGBeans(ConfigurationUtil.java:245)
>>>        at  
>>> org.apache.geronimo.kernel.config.KernelConfigurationManager.start 
>>> (KernelConfigurationManager.java:163)
>>>        at  
>>> org.apache.geronimo.kernel.config.SimpleConfigurationManager.startCo 
>>> nfiguration(SimpleConfigurationManager.java:331)
>>>        at  
>>> org.apache.geronimo.kernel.config.SimpleConfigurationManager$ 
>>> $FastClassByCGLIB$$ce77a924.invoke(<generated>)
>>>        at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53)
>>>        at  
>>> org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke 
>>> (FastMethodInvoker.java:38)
>>>        at org.apache.geronimo.gbean.runtime.GBeanOperation.invoke 
>>> (GBeanOperation.java:122)
>>>        at org.apache.geronimo.gbean.runtime.GBeanInstance.invoke 
>>> (GBeanInstance.java:813)
>>>        at org.apache.geronimo.gbean.runtime.RawInvoker.invoke 
>>> (RawInvoker.java:57)
>>>        at  
>>> org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke 
>>> (RawOperationInvoker.java:35)
>>>        at  
>>> org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept 
>>> (ProxyMethodInterceptor.java:96)
>>>        at org.apache.geronimo.kernel.config.ConfigurationManager$ 
>>> $EnhancerByCGLIB$$1b75b2b5.startConfiguration(<generated>)
>>>        at org.apache.geronimo.system.main.Daemon.doStartup 
>>> (Daemon.java:287)
>>>        at org.apache.geronimo.system.main.Daemon.<init> 
>>> (Daemon.java:74)
>>>        at org.apache.geronimo.system.main.Daemon.main(Daemon.java: 
>>> 367)
>>> started in 22/22   1s
>>> Startup completed in 25 seconds
>>> java.lang.IllegalArgumentException: Unable to look up connectors  
>>> for Tomcat container 'geronimo/tomcat/1.1-SNAPSHOT/car? 
>>> ServiceModule=geronimo/tomcat/1.1-SNAPSHOT/ 
>>> car,j2eeType=GBean,name=TomcatWebContainer':  
>>> java.lang.NullPointerException
>>>        at  
>>> org.apache.geronimo.tomcat.TomcatManagerImpl.getConnectorsForContain 
>>> er(TomcatManagerImpl.java:264)
>>>        at org.apache.geronimo.tomcat.TomcatManagerImpl$ 
>>> $FastClassByCGLIB$$6ff876df.invoke(<generated>)
>>>        at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53)
>>>        at  
>>> org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke 
>>> (FastMethodInvoker.java:38)
>>>        at org.apache.geronimo.gbean.runtime.GBeanOperation.invoke 
>>> (GBeanOperation.java:122)
>>>        at org.apache.geronimo.gbean.runtime.GBeanInstance.invoke 
>>> (GBeanInstance.java:848)
>>>        at org.apache.geronimo.kernel.basic.BasicKernel.invoke 
>>> (BasicKernel.java:244)
>>>        at  
>>> org.apache.geronimo.system.main.WebAppUtil.mapContainersToURLs 
>>> (WebAppUtil.java:52)
>>>        at  
>>> org.apache.geronimo.system.main.LongStartupMonitor.wrapUp 
>>> (LongStartupMonitor.java:195)
>>>        at  
>>> org.apache.geronimo.system.main.LongStartupMonitor.startupFinished 
>>> (LongStartupMonitor.java:104)
>>>        at org.apache.geronimo.system.main.Daemon.doStartup 
>>> (Daemon.java:314)
>>>        at org.apache.geronimo.system.main.Daemon.<init> 
>>> (Daemon.java:74)
>>>        at org.apache.geronimo.system.main.Daemon.main(Daemon.java: 
>>> 367)
>>>  Listening on Ports:
>>>    1099 0.0.0.0   RMI Naming
>>>    1389 0.0.0.0   Apache Directory LDAP
>>>    1527 0.0.0.0   Derby Connector
>>>    4201 127.0.0.1 ActiveIO Connector EJB
>>>    4242 127.0.0.1 Remote Login Listener
>>>    8009 0.0.0.0   Tomcat Connector AJP
>>>    8080 0.0.0.0   Tomcat Connector HTTP
>>>    8443 0.0.0.0   Tomcat Connector HTTPS
>>>    9999 0.0.0.0   JMX Remoting Connector
>>>   61616 0.0.0.0   ActiveMQ Message Broker Connector
>>>
>>>  Started Application Modules:
>>>    WAR: geronimo/jsp-examples-tomcat/1.1-SNAPSHOT/car
>>>    EAR: geronimo/daytrader-derby-tomcat/1.1-SNAPSHOT/car
>>>    WAR: geronimo/welcome-tomcat/1.1-SNAPSHOT/car
>>>    WAR: geronimo/servlets-examples-tomcat/1.1-SNAPSHOT/car
>>>    WAR: geronimo/ldap-demo-tomcat/1.1-SNAPSHOT/car
>>>    RAR: geronimo/activemq/1.1-SNAPSHOT/car
>>>    WAR: geronimo/jmxdebug-tomcat/1.1-SNAPSHOT/car
>>>    EAR: geronimo/webconsole-tomcat/1.1-SNAPSHOT/car
>>>    EAR: geronimo/uddi-tomcat/1.1-SNAPSHOT/car
>>>    RAR: geronimo/system-database/1.1-SNAPSHOT/car
>>>    WAR: geronimo/remote-deploy-tomcat/1.1-SNAPSHOT/car
>>>
>>>  Web Applications:
>>>    null/
>>>    null/console
>>>    null/console-standard
>>>    null/daytrader
>>>    null/debug-tool
>>>    null/jsp-examples
>>>    null/juddi
>>>    null/ldap-demo
>>>    null/remote-deploy
>>>    null/servlets-examples
>>>
>>> Geronimo Application Server started
>>> Server shutdown begun
>>> 09:30:44,124 WARN  [TcpTransportChannel] Peer closed connection
>>> java.io.EOFException
>>>        at java.io.DataInputStream.readByte(DataInputStream.java:333)
>>>        at org.activemq.io.AbstractWireFormat.readPacket 
>>> (AbstractWireFormat.java:230)
>>>        at org.activemq.transport.tcp.TcpTransportChannel.run 
>>> (TcpTransportChannel.java:313)
>>>        at java.lang.Thread.run(Thread.java:534)
>>> 09:30:44,130 WARN  [TcpTransportChannel] Peer closed connection
>>> java.io.EOFException
>>>        at java.io.DataInputStream.readByte(DataInputStream.java:333)
>>>        at org.activemq.io.AbstractWireFormat.readPacket 
>>> (AbstractWireFormat.java:230)
>>>        at org.activemq.transport.tcp.TcpTransportChannel.run 
>>> (TcpTransportChannel.java:313)
>>>        at java.lang.Thread.run(Thread.java:534)
>>> ^CFull thread dump Java HotSpot(TM) Client VM (1.4.2_10-b03 mixed  
>>> mode):
>>>
>>> "Thread-94" prio=7 tid=0x08706748 nid=0x91 waiting on condition  
>>> [c41d5000..c41d5d98]
>>>        at java.lang.Thread.sleep(Native Method)
>>>        at org.activemq.ra.ActiveMQAsfEndpointWorker.reconnect 
>>> (ActiveMQAsfEndpointWorker.java:177)
>>>        - locked <0xc7711290> (a  
>>> org.activemq.ra.ActiveMQAsfEndpointWorker)
>>>        at org.activemq.ra.ActiveMQAsfEndpointWorker.access$200 
>>> (ActiveMQAsfEndpointWorker.java:40)
>>>        at org.activemq.ra.ActiveMQAsfEndpointWorker$1.run 
>>> (ActiveMQAsfEndpointWorker.java:105)
>>>        - locked <0xc76fcc30> (a  
>>> org.activemq.ra.ActiveMQAsfEndpointWorker$1)
>>>        at org.apache.geronimo.connector.work.WorkerContext.run 
>>> (WorkerContext.java:291)
>>>        at EDU.oswego.cs.dl.util.concurrent.PooledExecutor 
>>> $Worker.run(Unknown Source)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "Thread-93" prio=7 tid=0x08771bd8 nid=0x90 in Object.wait()  
>>> [c09fd000..c09fdd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc64a7e28> (a java.lang.Object)
>>>        at EDU.oswego.cs.dl.util.concurrent.LinkedQueue.poll 
>>> (Unknown Source)
>>>        - locked <0xc64a7e28> (a java.lang.Object)
>>>        at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.getTask 
>>> (Unknown Source)
>>>        at EDU.oswego.cs.dl.util.concurrent.PooledExecutor 
>>> $Worker.run(Unknown Source)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "Thread-92" prio=7 tid=0x08250a20 nid=0x8f in Object.wait()  
>>> [c349d000..c349dd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc64a7e28> (a java.lang.Object)
>>>        at EDU.oswego.cs.dl.util.concurrent.LinkedQueue.poll 
>>> (Unknown Source)
>>>        - locked <0xc64a7e28> (a java.lang.Object)
>>>        at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.getTask 
>>> (Unknown Source)
>>>        at EDU.oswego.cs.dl.util.concurrent.PooledExecutor 
>>> $Worker.run(Unknown Source)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "Thread-91" prio=7 tid=0x0929ba18 nid=0x8e in Object.wait()  
>>> [c37ab000..c37abd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc64a7e28> (a java.lang.Object)
>>>        at EDU.oswego.cs.dl.util.concurrent.LinkedQueue.poll 
>>> (Unknown Source)
>>>        - locked <0xc64a7e28> (a java.lang.Object)
>>>        at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.getTask 
>>> (Unknown Source)
>>>        at EDU.oswego.cs.dl.util.concurrent.PooledExecutor 
>>> $Worker.run(Unknown Source)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "Thread-90" prio=7 tid=0x0853b4c8 nid=0x8d in Object.wait()  
>>> [c37ed000..c37edd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc64a7e28> (a java.lang.Object)
>>>        at EDU.oswego.cs.dl.util.concurrent.LinkedQueue.poll 
>>> (Unknown Source)
>>>        - locked <0xc64a7e28> (a java.lang.Object)
>>>        at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.getTask 
>>> (Unknown Source)
>>>        at EDU.oswego.cs.dl.util.concurrent.PooledExecutor 
>>> $Worker.run(Unknown Source)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "SIGINT handler" daemon prio=10 tid=0x0886ea08 nid=0x8a waiting  
>>> for monitor entry [c4193000..c4193d98]
>>>        at java.lang.Shutdown.exit(Shutdown.java:210)
>>>        - waiting to lock <0xca095c18> (a java.lang.Class)
>>>        at java.lang.Terminator$1.handle(Terminator.java:35)
>>>        at sun.misc.Signal$1.run(Signal.java:195)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "Geronimo shutdown thread" prio=5 tid=0x0876fe50 nid=0x78 waiting  
>>> for monitor entry [c09bb000..c09bbd98]
>>>        at org.activemq.ra.ActiveMQAsfEndpointWorker.stop 
>>> (ActiveMQAsfEndpointWorker.java:135)
>>>        - waiting to lock <0xc7711290> (a  
>>> org.activemq.ra.ActiveMQAsfEndpointWorker)
>>>        at  
>>> org.activemq.ra.ActiveMQResourceAdapter.endpointDeactivation 
>>> (ActiveMQResourceAdapter.java:261)
>>>        at  
>>> org.apache.geronimo.connector.ResourceAdapterWrapper.endpointDeactiv 
>>> ation(ResourceAdapterWrapper.java:92)
>>>        at org.apache.geronimo.connector.ResourceAdapterWrapper$ 
>>> $FastClassByCGLIB$$4ab28e73.invoke(<generated>)
>>>        at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53)
>>>        at  
>>> org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke 
>>> (FastMethodInvoker.java:38)
>>>        at org.apache.geronimo.gbean.runtime.GBeanOperation.invoke 
>>> (GBeanOperation.java:122)
>>>        at org.apache.geronimo.gbean.runtime.GBeanInstance.invoke 
>>> (GBeanInstance.java:813)
>>>        at org.apache.geronimo.gbean.runtime.RawInvoker.invoke 
>>> (RawInvoker.java:57)
>>>        at  
>>> org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke 
>>> (RawOperationInvoker.java:35)
>>>        at  
>>> org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept 
>>> (ProxyMethodInterceptor.java:96)
>>>        at org.apache.geronimo.connector.ResourceAdapterWrapper$ 
>>> $EnhancerByCGLIB$$d7c17cb8.endpointDeactivation(<generated>)
>>>        at  
>>> org.apache.geronimo.connector.ActivationSpecWrapper.deactivate 
>>> (ActivationSpecWrapper.java:109)
>>>        at org.apache.geronimo.connector.ActivationSpecWrapper$ 
>>> $FastClassByCGLIB$$aaa078c1.invoke(<generated>)
>>>        at net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53)
>>>        at  
>>> org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke 
>>> (FastMethodInvoker.java:38)
>>>        at org.apache.geronimo.gbean.runtime.GBeanOperation.invoke 
>>> (GBeanOperation.java:122)
>>>        at org.apache.geronimo.gbean.runtime.GBeanInstance.invoke 
>>> (GBeanInstance.java:813)
>>>        at org.apache.geronimo.gbean.runtime.RawInvoker.invoke 
>>> (RawInvoker.java:57)
>>>        at  
>>> org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke 
>>> (RawOperationInvoker.java:35)
>>>        at  
>>> org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept 
>>> (ProxyMethodInterceptor.java:96)
>>>        at org.apache.geronimo.connector.ActivationSpecWrapper$ 
>>> $EnhancerByCGLIB$$d908fdf6.deactivate(<generated>)
>>>        at org.openejb.mdb.MDBContainer.doStop(MDBContainer.java:226)
>>>        at  
>>> org.apache.geronimo.gbean.runtime.GBeanInstance.destroyInstance 
>>> (GBeanInstance.java:1139)
>>>        at  
>>> org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStop 
>>> (GBeanInstanceState.java:337)
>>>        at  
>>> org.apache.geronimo.gbean.runtime.GBeanInstanceState.stop 
>>> (GBeanInstanceState.java:188)
>>>        at org.apache.geronimo.gbean.runtime.GBeanInstance.stop 
>>> (GBeanInstance.java:554)
>>>        at org.apache.geronimo.kernel.basic.BasicKernel.stopGBean 
>>> (BasicKernel.java:435)
>>>        at  
>>> org.apache.geronimo.gbean.runtime.GBeanInstanceState.stop 
>>> (GBeanInstanceState.java:180)
>>>        at org.apache.geronimo.gbean.runtime.GBeanInstance.stop 
>>> (GBeanInstance.java:554)
>>>        at org.apache.geronimo.kernel.basic.BasicKernel.stopGBean 
>>> (BasicKernel.java:435)
>>>        at  
>>> org.apache.geronimo.gbean.runtime.GBeanInstanceState.stop 
>>> (GBeanInstanceState.java:180)
>>>        at org.apache.geronimo.gbean.runtime.GBeanInstance.stop 
>>> (GBeanInstance.java:554)
>>>        at org.apache.geronimo.kernel.basic.BasicKernel.stopGBean 
>>> (BasicKernel.java:435)
>>>        at  
>>> org.apache.geronimo.gbean.runtime.GBeanInstanceState.stop 
>>> (GBeanInstanceState.java:180)
>>>        at org.apache.geronimo.gbean.runtime.GBeanInstance.stop 
>>> (GBeanInstance.java:554)
>>>        at org.apache.geronimo.kernel.basic.BasicKernel.stopGBean 
>>> (BasicKernel.java:435)
>>>        at  
>>> org.apache.geronimo.gbean.runtime.GBeanInstanceState.stop 
>>> (GBeanInstanceState.java:180)
>>>        at org.apache.geronimo.gbean.runtime.GBeanInstance.stop 
>>> (GBeanInstance.java:554)
>>>        at org.apache.geronimo.kernel.basic.BasicKernel.stopGBean 
>>> (BasicKernel.java:435)
>>>        at  
>>> org.apache.geronimo.kernel.config.KernelConfigurationManager 
>>> $ShutdownHook.run(KernelConfigurationManager.java:261)
>>>        at  
>>> org.apache.geronimo.kernel.basic.BasicKernel.notifyShutdownHooks 
>>> (BasicKernel.java:685)
>>>        at org.apache.geronimo.kernel.basic.BasicKernel.shutdown 
>>> (BasicKernel.java:662)
>>>        at org.apache.geronimo.system.main.Daemon$1.run 
>>> (Daemon.java:245)
>>>
>>> "SIGINT handler" daemon prio=10 tid=0x09240c30 nid=0x75 in  
>>> Object.wait() [c440b000..c440bd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc631b018> (a  
>>> org.apache.geronimo.system.main.Daemon$1)
>>>        at java.lang.Thread.join(Thread.java:1001)
>>>        - locked <0xc631b018> (a  
>>> org.apache.geronimo.system.main.Daemon$1)
>>>        at java.lang.Thread.join(Thread.java:1054)
>>>        at java.lang.Shutdown.runHooks(Shutdown.java:130)
>>>        at java.lang.Shutdown.sequence(Shutdown.java:165)
>>>        at java.lang.Shutdown.exit(Shutdown.java:210)
>>>        - locked <0xca095c18> (a java.lang.Class)
>>>        at java.lang.Terminator$1.handle(Terminator.java:35)
>>>        at sun.misc.Signal$1.run(Signal.java:195)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "Thread-74" daemon prio=5 tid=0x0824fbb0 nid=0x73 waiting on  
>>> condition [c093d000..c093dd98]
>>>        at java.lang.Thread.sleep(Native Method)
>>>        at org.apache.geronimo.transaction.manager.TransactionTimer 
>>> $CurrentTime.run(TransactionTimer.java:38)
>>>
>>> "Geronimo Config Store Reaper" daemon prio=5 tid=0x0816bf18  
>>> nid=0x67 waiting on condition [c1edd000..c1eddd98]
>>>        at java.lang.Thread.sleep(Native Method)
>>>        at org.apache.geronimo.deployment.Deployer 
>>> $DeployerReaper.run(Deployer.java:365)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "TP-Monitor" daemon prio=5 tid=0x0848ec38 nid=0x66 in Object.wait 
>>> () [c1f9b000..c1f9bd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6b393d0> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $MonitorRunnable.run(ThreadPool.java:559)
>>>        - locked <0xc6b393d0> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "TP-Processor4" daemon prio=5 tid=0x0848dbe8 nid=0x65 runnable  
>>> [c1fdd000..c1fddd98]
>>>        at java.net.PlainSocketImpl.socketAccept(Native Method)
>>>        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
>>>        - locked <0xc6b16508> (a java.net.PlainSocketImpl)
>>>        at java.net.ServerSocket.implAccept(ServerSocket.java:448)
>>>        at java.net.ServerSocket.accept(ServerSocket.java:419)
>>>        at org.apache.jk.common.ChannelSocket.accept 
>>> (ChannelSocket.java:295)
>>>        at org.apache.jk.common.ChannelSocket.acceptConnections 
>>> (ChannelSocket.java:641)
>>>        at org.apache.jk.common.SocketAcceptor.runIt 
>>> (ChannelSocket.java:852)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:684)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "TP-Processor3" daemon prio=5 tid=0x0848d1a0 nid=0x64 in  
>>> Object.wait() [c201f000..c201fd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6b39580> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6b39580> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "TP-Processor2" daemon prio=5 tid=0x0848c758 nid=0x63 in  
>>> Object.wait() [c2061000..c2061d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6b39600> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6b39600> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "TP-Processor1" daemon prio=5 tid=0x0848c510 nid=0x62 in  
>>> Object.wait() [c20a3000..c20a3d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6b39680> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6b39680> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Monitor" prio=5 tid=0x080dee20 nid=0x61 in  
>>> Object.wait() [c20e5000..c20e5d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a57cf8> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $MonitorRunnable.run(ThreadPool.java:559)
>>>        - locked <0xc6a57cf8> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor25" daemon prio=5 tid=0x081427d8  
>>> nid=0x60 runnable [c2127000..c2127d98]
>>>        at java.net.PlainSocketImpl.socketAccept(Native Method)
>>>        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
>>>        - locked <0xc6a0e6b0> (a java.net.PlainSocketImpl)
>>>        at java.net.ServerSocket.implAccept(ServerSocket.java:448)
>>>        at java.net.ServerSocket.accept(ServerSocket.java:419)
>>>        at  
>>> org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket 
>>> (DefaultServerSocketFactory.java:60)
>>>        at org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket 
>>> (PoolTcpEndpoint.java:407)
>>>        at  
>>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt 
>>> (LeaderFollowerWorkerThread.java:70)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:684)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor24" daemon prio=5 tid=0x08141d90  
>>> nid=0x5f in Object.wait() [c2169000..c2169d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a57e00> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a57e00> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor23" daemon prio=5 tid=0x08141348  
>>> nid=0x5e in Object.wait() [c21ab000..c21abd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a57e80> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a57e80> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor22" daemon prio=5 tid=0x08140900  
>>> nid=0x5d in Object.wait() [c21ed000..c21edd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a57f00> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a57f00> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor21" daemon prio=5 tid=0x0813feb8  
>>> nid=0x5c in Object.wait() [c222f000..c222fd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a57f80> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a57f80> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor20" daemon prio=5 tid=0x0813f470  
>>> nid=0x5b in Object.wait() [c2271000..c2271d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58000> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58000> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor19" daemon prio=5 tid=0x0813ea28  
>>> nid=0x5a in Object.wait() [c22b3000..c22b3d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58080> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58080> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor18" daemon prio=5 tid=0x0813dfe0  
>>> nid=0x59 in Object.wait() [c22f5000..c22f5d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58100> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58100> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor17" daemon prio=5 tid=0x0813d598  
>>> nid=0x58 in Object.wait() [c2337000..c2337d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58180> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58180> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor16" daemon prio=5 tid=0x0813cb50  
>>> nid=0x57 in Object.wait() [c2379000..c2379d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58200> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58200> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor15" daemon prio=5 tid=0x0813c108  
>>> nid=0x56 in Object.wait() [c23bb000..c23bbd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58280> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58280> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor14" daemon prio=5 tid=0x0813b6c0  
>>> nid=0x55 in Object.wait() [c23fd000..c23fdd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58300> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58300> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor13" daemon prio=5 tid=0x0883a1e8  
>>> nid=0x54 in Object.wait() [c243f000..c243fd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58380> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58380> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor12" daemon prio=5 tid=0x08839198  
>>> nid=0x53 in Object.wait() [c2481000..c2481d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58400> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58400> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor11" daemon prio=5 tid=0x08838750  
>>> nid=0x52 in Object.wait() [c24c3000..c24c3d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58480> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58480> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor10" daemon prio=5 tid=0x08837d08  
>>> nid=0x51 in Object.wait() [c2505000..c2505d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58500> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58500> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor9" daemon prio=5 tid=0x088372c0  
>>> nid=0x50 in Object.wait() [c2547000..c2547d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58580> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58580> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor8" daemon prio=5 tid=0x08836878  
>>> nid=0x4f in Object.wait() [c2589000..c2589d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58600> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58600> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor7" daemon prio=5 tid=0x08835e30  
>>> nid=0x4e in Object.wait() [c25cb000..c25cbd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58680> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58680> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor6" daemon prio=5 tid=0x088353e8  
>>> nid=0x4d in Object.wait() [c260d000..c260dd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58700> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58700> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor5" daemon prio=5 tid=0x088349a0  
>>> nid=0x4c in Object.wait() [c264f000..c264fd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58780> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58780> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor4" daemon prio=5 tid=0x08833f58  
>>> nid=0x4b in Object.wait() [c2691000..c2691d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58800> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58800> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor3" daemon prio=5 tid=0x08833510  
>>> nid=0x4a in Object.wait() [c26d3000..c26d3d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58880> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58880> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor2" daemon prio=5 tid=0x082f5b08  
>>> nid=0x49 in Object.wait() [c2715000..c2715d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58900> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58900> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8080-Processor1" daemon prio=5 tid=0x082f58c0  
>>> nid=0x48 in Object.wait() [c2757000..c2757d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6a58980> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc6a58980> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Monitor" prio=5 tid=0x082f26e0 nid=0x47 in  
>>> Object.wait() [c2799000..c2799d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69ed640> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $MonitorRunnable.run(ThreadPool.java:559)
>>>        - locked <0xc69ed640> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor25" daemon prio=5 tid=0x082f1c98  
>>> nid=0x46 runnable [c27db000..c27dbd98]
>>>        at java.net.PlainSocketImpl.socketAccept(Native Method)
>>>        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
>>>        - locked <0xc69b4e58> (a java.net.PlainSocketImpl)
>>>        at java.net.ServerSocket.implAccept(ServerSocket.java:448)
>>>        at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.accept 
>>> (DashoA12275)
>>>        at  
>>> org.apache.tomcat.util.net.jsse.JSSESocketFactory.acceptSocket 
>>> (JSSESocketFactory.java:110)
>>>        at org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket 
>>> (PoolTcpEndpoint.java:407)
>>>        at  
>>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt 
>>> (LeaderFollowerWorkerThread.java:70)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:684)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor24" daemon prio=5 tid=0x082f0848  
>>> nid=0x45 in Object.wait() [c281d000..c281dd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f11b0> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f11b0> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor23" daemon prio=5 tid=0x082efe00  
>>> nid=0x44 in Object.wait() [c288f000..c288fd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f12a8> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f12a8> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor22" daemon prio=5 tid=0x082ef3b8  
>>> nid=0x43 in Object.wait() [c28d1000..c28d1d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f13a0> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f13a0> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor21" daemon prio=5 tid=0x082ee970  
>>> nid=0x42 in Object.wait() [c2913000..c2913d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f1498> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f1498> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor20" daemon prio=5 tid=0x088adb70  
>>> nid=0x41 in Object.wait() [c2955000..c2955d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f1590> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f1590> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor19" daemon prio=5 tid=0x088ad528  
>>> nid=0x40 in Object.wait() [c2997000..c2997d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f1688> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f1688> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor18" daemon prio=5 tid=0x088acae0  
>>> nid=0x3f in Object.wait() [c29d9000..c29d9d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f1780> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f1780> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor17" daemon prio=5 tid=0x088ac098  
>>> nid=0x3e in Object.wait() [c2a1b000..c2a1bd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f1878> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f1878> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor16" daemon prio=5 tid=0x088ab650  
>>> nid=0x3d in Object.wait() [c2a5d000..c2a5dd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f1970> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f1970> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor15" daemon prio=5 tid=0x088aac08  
>>> nid=0x3c in Object.wait() [c2a9f000..c2a9fd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f1a68> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f1a68> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor14" daemon prio=5 tid=0x088a9bb8  
>>> nid=0x3b in Object.wait() [c2ae1000..c2ae1d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f1b60> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f1b60> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor13" daemon prio=5 tid=0x088a9170  
>>> nid=0x3a in Object.wait() [c2b23000..c2b23d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f1c58> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f1c58> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor12" daemon prio=5 tid=0x088a8728  
>>> nid=0x39 in Object.wait() [c2b65000..c2b65d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f1d50> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f1d50> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor11" daemon prio=5 tid=0x088a7ce0  
>>> nid=0x38 in Object.wait() [c2ba7000..c2ba7d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f1e48> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f1e48> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor10" daemon prio=5 tid=0x088a7298  
>>> nid=0x37 in Object.wait() [c2be9000..c2be9d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f1f40> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f1f40> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor9" daemon prio=5 tid=0x088a6930  
>>> nid=0x36 in Object.wait() [c2c2b000..c2c2bd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f2038> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f2038> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor8" daemon prio=5 tid=0x0830f658  
>>> nid=0x35 in Object.wait() [c2c6d000..c2c6dd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f2130> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f2130> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor7" daemon prio=5 tid=0x0830f0f0  
>>> nid=0x34 in Object.wait() [c2caf000..c2cafd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f2228> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f2228> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor6" daemon prio=5 tid=0x084c8ec0  
>>> nid=0x33 in Object.wait() [c2cf1000..c2cf1d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f2320> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f2320> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor5" daemon prio=5 tid=0x084c8d58  
>>> nid=0x32 in Object.wait() [c2d33000..c2d33d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f2418> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f2418> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor4" daemon prio=5 tid=0x08310248  
>>> nid=0x31 in Object.wait() [c2d75000..c2d75d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f2510> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f2510> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor3" daemon prio=5 tid=0x084c9898  
>>> nid=0x30 in Object.wait() [c2db7000..c2db7d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f2608> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f2608> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor2" daemon prio=5 tid=0x084c9730  
>>> nid=0x2f in Object.wait() [c2df9000..c2df9d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f2700> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f2700> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "http-0.0.0.0-8443-Processor1" daemon prio=5 tid=0x0830f8e0  
>>> nid=0x2e in Object.wait() [c2e3b000..c2e3bd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc69f27f8> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.tomcat.util.threads.ThreadPool 
>>> $ControlRunnable.run(ThreadPool.java:656)
>>>        - locked <0xc69f27f8> (a  
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "ContainerBackgroundProcessor[StandardEngine[Geronimo]]" daemon  
>>> prio=5 tid=0x082b85c0 nid=0x2d waiting on condition  
>>> [c2e7d000..c2e7dd98]
>>>        at java.lang.Thread.sleep(Native Method)
>>>        at org.apache.catalina.core.ContainerBase 
>>> $ContainerBackgroundProcessor.run(ContainerBase.java:1546)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "Thread-7" daemon prio=5 tid=0x087bf340 nid=0x1c in Object.wait()  
>>> [c42dd000..c42ddd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6585648> (a java.util.TaskQueue)
>>>        at java.util.TimerThread.mainLoop(Timer.java:429)
>>>        - locked <0xc6585648> (a java.util.TaskQueue)
>>>        at java.util.TimerThread.run(Timer.java:382)
>>>
>>> "RMI LeaseChecker" daemon prio=5 tid=0x087521d8 nid=0x19 waiting  
>>> on condition [c43c9000..c43c9d98]
>>>        at java.lang.Thread.sleep(Native Method)
>>>        at sun.rmi.transport.DGCImpl$LeaseChecker.run(DGCImpl.java: 
>>> 307)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "RMI RenewClean-[172.21.35.85:9999]" daemon prio=5 tid=0x08410790  
>>> nid=0x17 in Object.wait() [c444d000..c444dd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6509f90> (a java.lang.ref.ReferenceQueue 
>>> $Lock)
>>>        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java: 
>>> 111)
>>>        - locked <0xc6509f90> (a java.lang.ref.ReferenceQueue$Lock)
>>>        at sun.rmi.transport.DGCClient$EndpointEntry 
>>> $RenewCleanThread.run(DGCClient.java:500)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "GC Daemon" daemon prio=2 tid=0x0876f2c0 nid=0x15 in Object.wait 
>>> () [c44f5000..c44f5d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc6506288> (a sun.misc.GC$LatencyLock)
>>>        at sun.misc.GC$Daemon.run(GC.java:100)
>>>        - locked <0xc6506288> (a sun.misc.GC$LatencyLock)
>>>
>>> "RMI Reaper" prio=5 tid=0x08415a18 nid=0x14 in Object.wait()  
>>> [c4537000..c4537d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc63fac28> (a java.lang.ref.ReferenceQueue 
>>> $Lock)
>>>        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java: 
>>> 111)
>>>        - locked <0xc63fac28> (a java.lang.ref.ReferenceQueue$Lock)
>>>        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java: 
>>> 127)
>>>        at sun.rmi.transport.ObjectTable$Reaper.run 
>>> (ObjectTable.java:330)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "RMI TCP Accept-9999" daemon prio=5 tid=0x0849d668 nid=0x13  
>>> runnable [c4579000..c4579d98]
>>>        at java.net.PlainSocketImpl.socketAccept(Native Method)
>>>        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
>>>        - locked <0xc6506338> (a java.net.PlainSocketImpl)
>>>        at java.net.ServerSocket.implAccept(ServerSocket.java:448)
>>>        at java.net.ServerSocket.accept(ServerSocket.java:419)
>>>        at sun.rmi.transport.tcp.TCPTransport.run 
>>> (TCPTransport.java:334)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "Socket Server: tcp://syisolx86vm1:4242" daemon prio=5  
>>> tid=0x084126f0 nid=0x12 runnable [c45bb000..c45bbd98]
>>>        at java.net.PlainSocketImpl.socketAccept(Native Method)
>>>        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
>>>        - locked <0xc6502430> (a java.net.PlainSocketImpl)
>>>        at java.net.ServerSocket.implAccept(ServerSocket.java:448)
>>>        at java.net.ServerSocket.accept(ServerSocket.java:419)
>>>        at org.activeio.net.SocketStreamChannelServer.accept 
>>> (SocketStreamChannelServer.java:66)
>>>        at org.activeio.net.SocketSyncChannelServer.accept 
>>> (SocketSyncChannelServer.java:48)
>>>        at org.activeio.adapter.SyncToAsyncChannelServer.run 
>>> (SyncToAsyncChannelServer.java:140)
>>>        at EDU.oswego.cs.dl.util.concurrent.PooledExecutor 
>>> $Worker.run(Unknown Source)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "LoginService login modules monitor" daemon prio=5 tid=0x08756c30  
>>> nid=0x11 in Object.wait() [c45fd000..c45fdd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc64fd090> (a  
>>> EDU.oswego.cs.dl.util.concurrent.ClockDaemon)
>>>        at EDU.oswego.cs.dl.util.concurrent.ClockDaemon.nextTask 
>>> (Unknown Source)
>>>        - locked <0xc64fd090> (a  
>>> EDU.oswego.cs.dl.util.concurrent.ClockDaemon)
>>>        at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run 
>>> (Unknown Source)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "service.EJB@11369856" daemon prio=5 tid=0x0838ab68 nid=0x10  
>>> runnable [c485d000..c485dd98]
>>>        at java.net.PlainSocketImpl.socketAccept(Native Method)
>>>        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
>>>        - locked <0xc64b2910> (a java.net.PlainSocketImpl)
>>>        at java.net.ServerSocket.implAccept(ServerSocket.java:448)
>>>        at java.net.ServerSocket.accept(ServerSocket.java:419)
>>>        at org.activeio.xnet.ServiceDaemon$SocketListener.run 
>>> (ServiceDaemon.java:146)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "FlushManager" daemon prio=5 tid=0x08387790 nid=0xf waiting on  
>>> condition [c4951000..c4951d98]
>>>        at java.lang.Thread.sleep(Native Method)
>>>        at org.objectweb.howl.log.LogBufferManager$FlushManager.run 
>>> (LogBufferManager.java:1232)
>>>
>>> "LogFileManager.EventManager" daemon prio=5 tid=0x0871bed8  
>>> nid=0xe in Object.wait() [c4993000..c4993d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc649aee0> (a java.lang.Object)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.objectweb.howl.log.LogFileManager$EventManager.run 
>>> (LogFileManager.java:1220)
>>>        - locked <0xc649aee0> (a java.lang.Object)
>>>
>>> "RMI TCP Accept-1099" daemon prio=5 tid=0x085b4cf8 nid=0xd  
>>> runnable [c4f79000..c4f79d98]
>>>        at java.net.PlainSocketImpl.socketAccept(Native Method)
>>>        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
>>>        - locked <0xc63ef478> (a java.net.PlainSocketImpl)
>>>        at java.net.ServerSocket.implAccept(ServerSocket.java:448)
>>>        at java.net.ServerSocket.accept(ServerSocket.java:419)
>>>        at sun.rmi.transport.tcp.TCPTransport.run 
>>> (TCPTransport.java:334)
>>>        at java.lang.Thread.run(Thread.java:534)
>>>
>>> "Thread-4" daemon prio=5 tid=0x085b4ae0 nid=0xc in Object.wait()  
>>> [c530b000..c530bd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc63ef550> (a java.util.TaskQueue)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at java.util.TimerThread.mainLoop(Timer.java:403)
>>>        - locked <0xc63ef550> (a java.util.TaskQueue)
>>>        at java.util.TimerThread.run(Timer.java:382)
>>>
>>> "Thread-2" daemon prio=5 tid=0x0841ac40 nid=0xb in Object.wait()  
>>> [c52c9000..c52c9d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc63b65f8> (a java.util.TaskQueue)
>>>        at java.util.TimerThread.mainLoop(Timer.java:429)
>>>        - locked <0xc63b65f8> (a java.util.TaskQueue)
>>>        at java.util.TimerThread.run(Timer.java:382)
>>>
>>> "Thread-0" prio=5 tid=0x08406458 nid=0x9 in Object.wait()  
>>> [c534d000..c534dd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc63a73b0> (a java.util.TaskQueue)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at java.util.TimerThread.mainLoop(Timer.java:403)
>>>        - locked <0xc63a73b0> (a java.util.TaskQueue)
>>>        at java.util.TimerThread.run(Timer.java:382)
>>>
>>> "Signal Dispatcher" daemon prio=10 tid=0x08112358 nid=0x6 waiting  
>>> on condition [0..d017bd58]
>>>
>>> "Finalizer" daemon prio=8 tid=0x081046f0 nid=0x4 in Object.wait()  
>>> [d07ed000..d07edd98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc62ef960> (a java.lang.ref.ReferenceQueue 
>>> $Lock)
>>>        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java: 
>>> 111)
>>>        - locked <0xc62ef960> (a java.lang.ref.ReferenceQueue$Lock)
>>>        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java: 
>>> 127)
>>>        at java.lang.ref.Finalizer$FinalizerThread.run 
>>> (Finalizer.java:159)
>>>
>>> "Reference Handler" daemon prio=10 tid=0x08103cd8 nid=0x3 in  
>>> Object.wait() [d0848000..d0848d98]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc62ef9c8> (a java.lang.ref.Reference$Lock)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at java.lang.ref.Reference$ReferenceHandler.run 
>>> (Reference.java:115)
>>>        - locked <0xc62ef9c8> (a java.lang.ref.Reference$Lock)
>>>
>>> "main" prio=5 tid=0x08072e90 nid=0x1 in Object.wait()  
>>> [8046000..8047030]
>>>        at java.lang.Object.wait(Native Method)
>>>        - waiting on <0xc630aa28> (a  
>>> org.apache.geronimo.kernel.basic.BasicKernel)
>>>        at java.lang.Object.wait(Object.java:429)
>>>        at org.apache.geronimo.system.main.Daemon.doStartup 
>>> (Daemon.java:321)
>>>        - locked <0xc630aa28> (a  
>>> org.apache.geronimo.kernel.basic.BasicKernel)
>>>        at org.apache.geronimo.system.main.Daemon.<init> 
>>> (Daemon.java:74)
>>>        at org.apache.geronimo.system.main.Daemon.main(Daemon.java: 
>>> 367)
>>>
>>> "VM Thread" prio=5 tid=0x08102ec0 nid=0x2 runnable
>>>
>>> "VM Periodic Task Thread" prio=10 tid=0x08115bd0 nid=0x8 waiting  
>>> on condition
>>> "Suspend Checker Thread" prio=10 tid=0x08111a20 nid=0x5 runnable
>>>
>>
>>
>


Mime
View raw message