activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joe Kutner (JIRA)" <j...@apache.org>
Subject [jira] Created: (AMQ-1599) Client deadlock after IllegalMonitorStateException on server
Date Mon, 25 Feb 2008 17:35:17 GMT
Client deadlock after IllegalMonitorStateException on server
------------------------------------------------------------

                 Key: AMQ-1599
                 URL: https://issues.apache.org/activemq/browse/AMQ-1599
             Project: ActiveMQ
          Issue Type: Bug
          Components: Broker
    Affects Versions: 4.1.1
         Environment: Solaris , Others
            Reporter: Joe Kutner


The server threw an IllegalMonitorStateException from:
org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:386)

This resulted in the FutureResponse on the client never reveiving a result.  Thus, the CountDownLatch
in org.apache.activeio.journal.active.BatchedWrite never received a 'countDown()'.  This results
in a the remaining Threads to block at:
org.apache.activeio.journal.active.JournalImpl.write(JournalImpl.java:171)

Here is the stack trace and log from the server:

---

DEBUG - 21/2 08:56:41 - Checkpoint started. - org.apache.activemq.store.journal.JournalPersistenceAdapter
336
ERROR - 21/2 08:56:41 - Failed to checkpoint a message store: java.lang.IllegalMonitorStateException
- org.apache.activemq.store.journal.JournalPersistenceAdapter 395
java.lang.IllegalMonitorStateException
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at edu.emory.mathcs.backport.java.util.concurrent.FutureTask.waitFor(FutureTask.java:267)
	at edu.emory.mathcs.backport.java.util.concurrent.FutureTask.get(FutureTask.java:117)
	at org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:386)
	at org.apache.activemq.store.journal.JournalPersistenceAdapter$2.iterate(JournalPersistenceAdapter.java:129)
	at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:117)
	at org.apache.activemq.thread.PooledTaskRunner.access$100(PooledTaskRunner.java:26)
	at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:44)
	at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
	at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
	at java.lang.Thread.run(Thread.java:619)
DEBUG - 21/2 08:56:41 - Marking journal at: 2:7588766 - org.apache.activemq.store.journal.JournalPersistenceAdapter
402
ERROR - 21/2 08:56:41 - Failed to mark the Journal: java.io.IOException: Write failed: java.lang.IllegalMonitorStateException
- org.apache.activemq.store.journal.JournalPersistenceAdapter 407
java.io.IOException: Write failed: java.lang.IllegalMonitorStateException
	at org.apache.activeio.journal.active.JournalImpl.write(JournalImpl.java:197)
	at org.apache.activeio.journal.active.JournalImpl.setMark(JournalImpl.java:345)
	at org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:403)
	at org.apache.activemq.store.journal.JournalPersistenceAdapter$2.iterate(JournalPersistenceAdapter.java:129)
	at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:117)
	at org.apache.activemq.thread.PooledTaskRunner.access$100(PooledTaskRunner.java:26)
	at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:44)
	at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
	at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
	at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.IllegalMonitorStateException
	at java.lang.Object.notify(Native Method)
	at org.apache.activeio.journal.active.BatchedWrite.append(BatchedWrite.java:85)
	at org.apache.activeio.journal.active.JournalImpl.addToPendingWriteBatch(JournalImpl.java:222)
	at org.apache.activeio.journal.active.JournalImpl.write(JournalImpl.java:180)
	... 9 more

---

Here are the stack traces of the threads that were waiting:

---

Name: ActiveMQ Transport: tcp:///192.149.134.237:36752
State: WAITING on edu.emory.mathcs.backport.java.util.concurrent.CountDownLatch@3952621c
Total blocked: 1  Total waited: 1

Stack trace:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:485)
edu.emory.mathcs.backport.java.util.concurrent.CountDownLatch.await(CountDownLatch.java:179)
org.apache.activeio.journal.active.BatchedWrite.waitForForce(BatchedWrite.java:96)
org.apache.activeio.journal.active.JournalImpl.write(JournalImpl.java:188)
org.apache.activeio.journal.active.JournalImpl.write(JournalImpl.java:158)
org.apache.activemq.store.journal.JournalPersistenceAdapter.writeCommand(JournalPersistenceAdapter.java:596)
org.apache.activemq.store.journal.JournalTransactionStore.commit(JournalTransactionStore.java:178)
org.apache.activemq.transaction.LocalTransaction.commit(LocalTransaction.java:68)
org.apache.activemq.broker.TransactionBroker.commitTransaction(TransactionBroker.java:154)
org.apache.activemq.broker.BrokerFilter.commitTransaction(BrokerFilter.java:94)
org.apache.activemq.broker.BrokerFilter.commitTransaction(BrokerFilter.java:94)
org.apache.activemq.broker.BrokerFilter.commitTransaction(BrokerFilter.java:94)
org.apache.activemq.broker.MutableBrokerFilter.commitTransaction(MutableBrokerFilter.java:104)
org.apache.activemq.broker.TransportConnection.processCommitTransactionOnePhase(TransportConnection.java:424)
   - locked org.apache.activemq.broker.jmx.ManagedTransportConnection@1c22c954
org.apache.activemq.command.TransactionInfo.visit(TransactionInfo.java:99)
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:294)
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:185)
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:65)
org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:133)
org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:122)
org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:137)
java.lang.Thread.run(Thread.java:619)

---

Name: ActiveMQ Transport: tcp:///192.149.134.238:37558
State: BLOCKED on org.apache.activeio.journal.active.JournalImpl@1b93c5f owned by: Persistence
Adaptor Task
Total blocked: 2  Total waited: 0

Stack trace: 
org.apache.activeio.journal.active.JournalImpl.write(JournalImpl.java:171)
org.apache.activeio.journal.active.JournalImpl.write(JournalImpl.java:158)
org.apache.activemq.store.journal.JournalPersistenceAdapter.writeCommand(JournalPersistenceAdapter.java:596)
org.apache.activemq.store.journal.JournalMessageStore.addMessage(JournalMessageStore.java:95)
org.apache.activemq.broker.region.Queue.sendMessage(Queue.java:333)
org.apache.activemq.broker.region.Queue.send(Queue.java:326)
org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:292)
org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:382)
org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:193)
org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:126)
org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:98)
org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:126)
org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:136)
org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:498)
org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:604)
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:294)
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:185)
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:65)
org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:133)
org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:122)
org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:137)
java.lang.Thread.run(Thread.java:619)

----

Name: pool-flow.seda.gov.nasa.core.common.bc.persistence-1.0-thread-1489
State: WAITING on edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar@131653cd
Total blocked: 1  Total waited: 4

Stack trace: 
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:485)
edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar.await(CondVar.java:75)
edu.emory.mathcs.backport.java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:318)
org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:42)
org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:75)
org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1171)
org.apache.activemq.TransactionContext.commit(TransactionContext.java:260)
org.apache.activemq.ActiveMQSession.commit(ActiveMQSession.java:464)
gov.nasa.core.common.bc.persistence.jms.JMSProducer.commit(JMSProducer.java:51)
gov.nasa.core.common.bc.persistence.jms.ESBProducerWrapper.send(ESBProducerWrapper.java:55)
gov.nasa.core.common.bc.persistence.PersistenceComponent.produceMessage(PersistenceComponent.java:180)
gov.nasa.core.common.bc.persistence.PersistenceComponent.onActive(PersistenceComponent.java:93)
gov.nasa.esb.framework.jbi.AbstractNASAComponent.onMessageExchange(AbstractNASAComponent.java:157)
org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImpl.java:593)
org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:174)
org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:176)
org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
java.lang.Thread.run(Thread.java:619)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message