activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Riccardi (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMQCPP-393) TimeToLive feature makes some messages disappear
Date Tue, 17 Apr 2012 13:23:18 GMT

    [ https://issues.apache.org/jira/browse/AMQCPP-393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255549#comment-13255549
] 

Thomas Riccardi commented on AMQCPP-393:
----------------------------------------

The issue is visible here in the logs:
{noformat}
2012-04-06 10:30:12,810 | TRACE | ack:MessageAck {commandId = 19, responseRequired = false,
ackType = 2, consumerId = ID:CLM3X4J-45976-1333700962062-0:0:0:0, firstMessageId = ID:CLM3X4J-59268-1333700662848-0:0:0:10:3,
lastMessageId = ID:CLM3X4J-59268-1333700662848-0:0:0:10:96, destination = queue://Request.c92bb686-74e9-47a0-8865-1a3900939387,
transactionId = TX:ID:CLM3X4J-45976-1333700962062-0:0:3, messageCount = 94, poisonCause =
null} | org.apache.activemq.broker.region.PrefetchSubscription | ActiveMQ Transport: tcp:///127.0.0.1:59987
{noformat}

94 messages are acked with ackType=2 (ACK_TYPE_CONSUMED), but there should be only one message
acked with type 2 (message id ID:CLM3X4J-59268-1333700662848-0:0:0:10:3), the 93 others are
the 93 lost messages: they were prefetched by the consumer (but why? prefetch is set to 0...),
and internally marked expired by the consumer.

They should either be acked with type 0 (ACK_TYPE_DELIVERED), or not acked at all.
This explains the issue: the messages expiration is checked by the scheduler which runs from
time to time, and in this case, the messages seen as expired by the consumer are not yet marked
as expired by the broker. Then the broker receives a "consumed" ack for those messages: this
is what makes them disappear.


By comparing java and c++ client codes, I was able to find a difference: this commit on the
java client has no equivalent in CMS:
http://svn.apache.org/viewvc/activemq/trunk/activemq-core/src/main/java/org/apache/activemq/ActiveMQMessageConsumer.java?r1=777462&r2=777463

This is related to this bug: https://issues.apache.org/jira/browse/AMQ-2265

In our case the expired messages are pushed into _dispatchedMessages_ list, and _ActiveMQConsumer::makeAckForAllDeliveredMessages()_
will ack all messages from _dispatchedMessages_ with a forced ackType.


When porting the modifications from java to CMS, no more false ack happens, and the bug disappears.
                
> TimeToLive feature makes some messages disappear
> ------------------------------------------------
>
>                 Key: AMQCPP-393
>                 URL: https://issues.apache.org/jira/browse/AMQCPP-393
>             Project: ActiveMQ C++ Client
>          Issue Type: Bug
>          Components: CMS Impl
>    Affects Versions: 3.4.0
>         Environment: Ubuntu 10.04 x86_64, CMS 3.4.0, ActiveMQ 5.5.0 (and 5.6 SNAPSHOT
of 2012-04-05)
>            Reporter: Thomas Riccardi
>            Assignee: Timothy Bish
>         Attachments: activemq-cpp-library-3.4.1-src.fix-prefetched-expired.patch, activemq.log.gz
>
>
> The following setup is run in one machine:
> ActiveMQ Broker 5.5.0, with:
> {code:xml}
>                   <deadLetterStrategy>
>                     <sharedDeadLetterStrategy 
>                         processExpired="true"
>                         processNonPersistent="true"
>                         />
>                   </deadLetterStrategy>
> {code}
> Two C++ programs using CMS 3.4.0:
> Program A produces messages to a "request" queue with a TTL of 1000ms.
> Program B consumes messages from "request" queue with prefetch=0, process the messages,
and posts a reply in "reply" queue.
> Program A sends 222 messages to the "request" queue.
> Program B process a message in about 250ms.
> Program B has the time to process 4 messages from "request", and post the 4 replies to
"reply".
> Then the broker moves to the "DLQ" queue the expired messages from the "request" queue.
> Final state: "request" is empty, "reply" has 4 enqueued messages, and "DLQ" has 125 enqueued
messages.
> *The bug*: the sum of messages counts from "DLQ" and "reply" is lower than the number
of messages initially enqueued in "request": there are 93 messages missing.
> The messages are not lost by the Program B: with traces on the receive() loop we confirm
that it only got 4 messages.
> {code}
>       while (1)
>       {
>         cms::Message *message = _messageConsumer->receive();
>         if (! message)
>           continue;
>         _messageListener->onMessage(message);
>         delete message;
>       }
> {code}
> The bug is reproducible, but the numbers vary. This happens less often if we increase
the TTL to 10s, and this never happens with no TTL (TTL=0, default).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message