activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Torsten Mielke (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (AMQ-2009) Problem with message dispatch after a while
Date Wed, 18 Mar 2009 18:08:43 GMT

    [ https://issues.apache.org/activemq/browse/AMQ-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=50652#action_50652
] 

Torsten Mielke edited comment on AMQ-2009 at 3/18/09 11:08 AM:
---------------------------------------------------------------

I took consumertest.zip and turned it into a Maven JUnit test (see attached testcase.zip)
in order to investigate into this problem. This new testcase also reproduces the same behavior
however I do not believe this is a bug. Let me explain why.

Both consumers listen on the same queue. 
The first consumer only closes its session *after* the second consumer tried to receive the
message that the second producer sent to the queue. So the first consumer is still active
when the second consumer calls f_consumer.receive(5000);

So what happens at runtime is that even though both consumers use a pull mode (by calling
consumer.receive(5000); ) there is a default prefetch size (as this is not set explicitly)
that is used for each consumer. 
The first consumer acked the first message so it is available to receive more messages (even
though it does not actively call f_consumer.receive()). So when the second message appears
on the queue, the broker sends it right to the *first consumer* where it stays in the prefetch
queue until it either gets received by the consumer calling f_consumer.receive() or the session
gets closed. If in the testcase you call 
{code:java}
consumerTest.receiveMessage(dinges);
{code}

rather than 
{code:java}
consumerTestNew.receiveMessage(dinges);
{code}
the message is received fine. 

So there are two ways to work around this:

1. have the first consumer close its session before the second consumer receives the message:
{code:title=ConsumerProblemTest.java}
...
consumerTest.close();

//create second consumer and read msg
DurableConsumer consumerTestNew = new DurableConsumer();
consumerTestNew.init();
consumerTestNew.receiveMessage(dinges);
consumerTestNew.close();
{code}


2. use a prefetch limit of 0 so messages to not get prefetched to consumers:
{code:title=DurableConsumerjava}
env.put(InitialContext.PROVIDER_URL, "tcp://localhost:61616?jms.prefetchPolicy.queuePrefetch=0");
{code}

With any of the two changes the testcase succeeds. Give it a go. Simply run mvn test, it should
fail out of the box. Then make these changes, the testcase will succeed.

      was (Author: tmielke):
    I took consumertest.zip and turned it into a Maven JUnit test (see attached testcase.zip)
in order to investigate into this problem. This new testcase also reproduces the same behavior
however I do not believe this is a bug. Let me explain why.

Both consumers listen on the same queue. 
The first consumer only closes its session *after* the second consumer tried to receive the
message that the second producer sent to the queue. So the first consumer is still active
when the second consumer calls f_consumer.receive(5000);

So what happens at runtime is that even though both consumers use a pull mode (by calling
consumer.receive(5000);) there is a default prefetch size (as this is not set explicitly)
that is used for each consumer. 
The first consumer acked the first message so it is available to receive more messages (even
though it does not actively call f_consumer.receive()). So when the second message appears
on the queue, the broker sends it right to the *first consumer* where it stays in the prefetch
queue until it either gets received by the consumer calling f_consumer.receive() or the session
gets closed. If in the testcase you call 
{code:java}
consumerTest.receiveMessage(dinges);
{code}

rather than 
{code:java}
consumerTestNew.receiveMessage(dinges);
{code}
the message is received fine. 

So there are two ways to work around this:

1. have the first consumer close its session before the second consumer receives the message:
{code:title=ConsumerProblemTest.java}
...
consumerTest.close();

//create second consumer and read msg
DurableConsumer consumerTestNew = new DurableConsumer();
consumerTestNew.init();
consumerTestNew.receiveMessage(dinges);
consumerTestNew.close();
{code}


2. use a prefetch limit of 0 so messages to not get prefetched to consumers:
{code:title=DurableConsumerjava}
env.put(InitialContext.PROVIDER_URL, "tcp://localhost:61616?jms.prefetchPolicy.queuePrefetch=0");
{code}

With any of the two changes the testcase succeeds. Give it a go. Simply run mvn test, it should
fail out of the box. Then make these changes, the testcase will succeed.
  
> Problem with message dispatch after a while
> -------------------------------------------
>
>                 Key: AMQ-2009
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2009
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.1.0, 5.2.0
>            Reporter: Rajani Chennamaneni
>            Assignee: Rob Davies
>            Priority: Blocker
>         Attachments: consumertest.zip, DispatchMultipleConsumersTest.java, JConsole-screenshot.jpg,
testcase.zip
>
>
> Messages are not getting dispatched after a while (although it accepts new incoming messages)
until restart of the broker. This problem is described in several posts.
> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
> http://www.nabble.com/Stuck-messages---Dispatch-issues-td20467949.html
> There was also an issue opened in Spring project for this thinking it was Spring problem.
> http://jira.springframework.org/browse/SPR-5110
> I am not able to reproduce with Junit test case having BrokerService started with in
the test case. I guess I am not hitting the right stress conditions this way. But when I run
the test case against an externally running ActiveMQ instance backed with oracle database
persistence, it is reproducible most of the times. This is not a every time failure situation,
it takes more time once than the other.
> I was able to hit this situation of stuck messages on queue using following scenario
most of the times:
> 1) Start 2 concurrent consumers for the queue using Spring's DefaultMessageListenerContainer
using cacheLevelName as CACHE_CONSUMER
> 2) Send messages using JMETER 2.3.2 to the queue on ActiveMQ stand alone broker instance
with 50 threads looping 20 times.
> 3) After a while, you will notice that Spring logs that no messages are being received
but the messages are shown jconsole of ActiveMQ and the database backing it for persistence.
> But in 5.2 RC3, the problem is that it dispatches duplicate messages and does not remove
them from broker's database after acknowledge properly.
> Attached test case might help to reproduce when run against externally running stand
alone ActiveMQ broker. Another way to see the problem is that try to load test using JMETER
by sending messages to a queue with a camel route that moves messages from this queue to another
and you will notice that it stops moving after while or copied duplicates in case of 5.2 RC3.
> Sorry about such a huge description but it is a real problem! A different team at our
company are having this issue in production with 5.1. They are using it as an embedded broker
with derby for persistence.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message