activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Spidim <sdi...@gmail.com>
Subject Queue consumer stuck after failover reconnect to broker that had been restarted
Date Wed, 16 Nov 2016 16:36:15 GMT
I have setup a simple job queue using ActiveMQ and JMS in Java, that uses a
broker, a producer and a consumer with multiple threads. The consumer is set
up to use 3 threads and a MessageListener function, so 3 clients are
connected to broker and consume messages in round robin. Because messages
are slow to process (between a couple of seconds to some minutes) the policy
prefetch option is set to 1 in consumers.
 In general, everything works as expected, until the broker is restarted
(manually or after a failure) and the failover protocol reconnects the
consumers to broker. Usually, between the disconnection and the reconnection
the consumer might try to send a session.commit() or session.rollback()
(session is transacted), but this fails and a message of automatic rollback
by the AMQ is logged. Also after the reconnect, a
TransactionRolledBackException is triggered, which I handle in the
application code, by rolling back application logic and exiting message
handler function. 
What I would expect at this point, is the message to redispatch to a
consumer for reprocessing. But what happens is that consumers freeze
indefinitely and do not receive any more messages from the queue, old or new
messages, although the queue is not empty and the consumers are listed as
active consumers of the queue. The only difference I could find, after I
checked the status in the AMQ Web Admin GUI, is that the prefetch max value
is now 0 (before the failover was 1). Only after a restart of the Consumer
process, the normal consumption of queue messages is recovered. The
persistence store is KahaDB and only one broker is active. The destination
url used in consumers is:
failover:(tcp://brokernode:61616)?jms.prefetchPolicy.queuePrefetch=1&startupMaxReconnectAttempts=-1&maxReconnectAttempts=-1&maxReconnectDelay=3000&timeout=10000
I've tested different parameters for example
failover:(tcp://brokernode:61616)?jms.prefetchPolicy.queuePrefetch=1&startupMaxReconnectAttempts=-1&maxReconnectAttempts=-1&maxReconnectDelay=3000&timeout=10000&trackMessages=true&jms.redeliveryPolicy.maximumRedeliveryDelay=10000&jms.redeliveryPolicy.useExponentialBackOff=true&jms.redeliveryPolicy.useCollisionAvoidance=true
or
failover:(tcp://localhost:61616?wireFormat.maxInactivityDuration=0&keepAlive=true)?jms.prefetchPolicy.all=1&initialReconnectDelay=10000&useExponentialBackOff=false&maxReconnectAttempts=-1&randomize=false
But no luck, nothing seems to wake up consumers after the failover to start
consuming messages again. I've tried to manually receive a message using
consumer.receive(), but as expected I got an error message that while a
message listener is set, cannot use the receive method.
My question is, do I have to follow a specific procedure to activate
consumers again after a broker failover reconnection? For example close the
session and restart it or something else? Or is a configuration issue and I
have to set a configuration parameter in URL or activemq.xml file in broker?




--
View this message in context: http://activemq.2283324.n4.nabble.com/Queue-consumer-stuck-after-failover-reconnect-to-broker-that-had-been-restarted-tp4719264.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message