activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Tully (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (AMQ-1824) Single producer and multiple consumer processes, some consumers stop getting messages.
Date Mon, 03 Nov 2008 12:05:06 GMT

     [ https://issues.apache.org/activemq/browse/AMQ-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gary Tully resolved AMQ-1824.
-----------------------------

    Resolution: Working as Designed

This test is a demonstration of a limitation of the current geronimo transaction manager xid
factory. With three JMVs on the same machine executing the same code, the transaction identifiers
are identical.
The broker sees the same Xid on three connections instead of three separate transactions.
The problem occurs when a transaction commit occurs before the last ack on a connection.

The default geronimo xidFactory impl needs to add some entropy in the form of a random to
the baseId. The current use of HashCode and IP address for baseId is not sufficient. Typically
nodes will be on different IP addresses so this will not be an issue.
I raised a geronimo issue with a suggested patch:  https://issues.apache.org/jira/browse/GERONIMO-4393

As a workaround:
there is a constructor argument that allows the baseId to be specified, if this makes use
of the clientId property, the test works as expected.

{code}
  <bean id="xidFactory" class="org.apache.geronimo.transaction.manager.XidFactoryImpl">
  	<constructor-arg><value>bla-bla-${clientId}</value></constructor-arg>
  </bean>

  <bean id="transactionManager" class="org.jencks.factory.TransactionManagerFactoryBean"
lazy-init="default" autowire="default" dependency-check="default" >
  	<property name="xidFactory" ref="xidFactory" />
  </bean>
{code}

> Single producer and multiple consumer processes, some consumers stop getting messages.
> --------------------------------------------------------------------------------------
>
>                 Key: AMQ-1824
>                 URL: https://issues.apache.org/activemq/browse/AMQ-1824
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.1.0
>         Environment: Windows XP SP2, Sun JDK 1.6.03 and RHEL 64 bit Sun 1.6.05
>            Reporter: Jerry Shea
>            Assignee: Gary Tully
>            Priority: Blocker
>             Fix For: 5.3.0
>
>         Attachments: repro_AMQ.zip, repro_AMQ_rf.zip
>
>
> With a single producer and multiple consumer processes using the ActiveMQResourceAdapter
(inside Jencks), some consumers stop getting messages. If you look at the JMX counters inside
jconsole you can see the stuck subscription's pending queue size increasing. Nearly every
time this exception is thrown by one of the failing consumers:
> {noformat}
> 2008-06-25 12:07:29,297 [pool-1-thread-7] WARN  Transaction  - Error ending association
for XAResource org.apache.geronimo.transaction.manager.WrapperNamedXAResource@17918f0; transaction
will roll back. XA error code: -7
> javax.transaction.xa.XAException: The connection is already closed
> 	at org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:624)
> 	at org.apache.activemq.TransactionContext.setXid(TransactionContext.java:559)
> 	at org.apache.activemq.TransactionContext.end(TransactionContext.java:339)
> 	at org.apache.activemq.ra.LocalAndXATransaction.end(LocalAndXATransaction.java:90)
> 	at org.apache.geronimo.transaction.manager.WrapperNamedXAResource.end(WrapperNamedXAResource.java:51)
> 	at org.apache.geronimo.transaction.manager.TransactionImpl.endResources(TransactionImpl.java:563)
> 	at org.apache.geronimo.transaction.manager.TransactionImpl.endResources(TransactionImpl.java:542)
> 	at org.apache.geronimo.transaction.manager.TransactionImpl.beforePrepare(TransactionImpl.java:401)
> 	at org.apache.geronimo.transaction.manager.TransactionImpl.commit(TransactionImpl.java:257)
> 	at org.apache.geronimo.transaction.manager.TransactionManagerImpl.commit(TransactionManagerImpl.java:238)
> 	at org.jencks.XAEndpoint.afterDelivery(XAEndpoint.java:105)
> 	at org.apache.activemq.ra.MessageEndpointProxy$MessageEndpointAlive.afterDelivery(MessageEndpointProxy.java:121)
> 	at org.apache.activemq.ra.MessageEndpointProxy.afterDelivery(MessageEndpointProxy.java:63)
> 	at org.apache.activemq.ra.ServerSessionImpl.afterDelivery(ServerSessionImpl.java:217)
> 	at org.apache.activemq.ActiveMQSession.run(ActiveMQSession.java:785)
> 	at org.apache.activemq.ra.ServerSessionImpl.run(ServerSessionImpl.java:165)
> 	at org.apache.geronimo.connector.work.WorkerContext.run(WorkerContext.java:290)
> 	at org.apache.geronimo.connector.work.pool.NamedRunnable.run(NamedRunnable.java:32)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> 	at java.lang.Thread.run(Unknown Source)
> Caused by: org.apache.activemq.ConnectionClosedException: The connection is already closed
> 	at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1271)
> 	at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1258)
> 	at org.apache.activemq.TransactionContext.setXid(TransactionContext.java:556)
> 	... 19 more
> 2008-06-25 12:07:29,297 [pool-1-thread-7] ERROR Transaction  - Unexpected exception rolling
back org.apache.geronimo.transaction.manager.WrapperNamedXAResource@17918f0; continuing with
rollback
> javax.transaction.xa.XAException: The connection is already closed
> 	at org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:624)
> 	at org.apache.activemq.TransactionContext.rollback(TransactionContext.java:423)
> 	at org.apache.activemq.ra.LocalAndXATransaction.rollback(LocalAndXATransaction.java:128)
> 	at org.apache.geronimo.transaction.manager.WrapperNamedXAResource.rollback(WrapperNamedXAResource.java:78)
> 	at org.apache.geronimo.transaction.manager.TransactionImpl.rollbackResources(TransactionImpl.java:581)
> 	at org.apache.geronimo.transaction.manager.TransactionImpl.commit(TransactionImpl.java:267)
> 	at org.apache.geronimo.transaction.manager.TransactionManagerImpl.commit(TransactionManagerImpl.java:238)
> 	at org.jencks.XAEndpoint.afterDelivery(XAEndpoint.java:105)
> 	at org.apache.activemq.ra.MessageEndpointProxy$MessageEndpointAlive.afterDelivery(MessageEndpointProxy.java:121)
> 	at org.apache.activemq.ra.MessageEndpointProxy.afterDelivery(MessageEndpointProxy.java:63)
> 	at org.apache.activemq.ra.ServerSessionImpl.afterDelivery(ServerSessionImpl.java:217)
> 	at org.apache.activemq.ActiveMQSession.run(ActiveMQSession.java:785)
> 	at org.apache.activemq.ra.ServerSessionImpl.run(ServerSessionImpl.java:165)
> 	at org.apache.geronimo.connector.work.WorkerContext.run(WorkerContext.java:290)
> 	at org.apache.geronimo.connector.work.pool.NamedRunnable.run(NamedRunnable.java:32)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> 	at java.lang.Thread.run(Unknown Source)
> Caused by: org.apache.activemq.ConnectionClosedException: The connection is already closed
> 	at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1271)
> 	at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1258)
> 	at org.apache.activemq.TransactionContext.rollback(TransactionContext.java:407)
> 	... 16 more
> 2008-06-25 12:07:29,297 [pool-1-thread-7] INFO  org.apache.activemq.ra.ServerSessionImpl:0
 - Endpoint failed to process message. Reason: java.lang.RuntimeException: Endpoint after
delivery notification failure
> {noformat}
> I've written some code to reproduce this problem. Instructions:
> 1. unpack the attached source
> 2. build with 'mvn compile package'
> 3. start with ./startAll.sh (I used cygwin) - this will start 1 producer and 3 consumer
processes. These programs are pointing to tcp://localhost:62616 (you can change this in spring
config file)
> Look in jconsole under subscriptions and you will probably see within 1 or 2 runs, only
2 consumers being created, or possibly 3 consumers but 1 of them with an increasing pending
queue size.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message