activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Tully (JIRA)" <j...@apache.org>
Subject [jira] Commented: (AMQ-2803) 'Zombie' messages created in KahaDB after failover, with warning 'Duplicate message add attempt rejected.'
Date Mon, 28 Jun 2010 11:56:52 GMT

    [ https://issues.apache.org/activemq/browse/AMQ-2803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=60310#action_60310
] 

Gary Tully commented on AMQ-2803:
---------------------------------

I am working on it atm, so should have some completion by the end of the week. Any tests you
can provide to validate will be appreciated... there a a bunch of tests that assert duplicate
suppression and ordering but non that tie it together with the queue sizes  and jmx.
You can access the brokerview and pull out the relevant stats from junit test if u want to
verify them.

> 'Zombie' messages created in KahaDB after failover, with warning 'Duplicate message add
attempt rejected.'
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-2803
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2803
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Message Store
>    Affects Versions: 5.3.2
>            Reporter: Adrian Trenaman
>
> Am doing some failover testing of ActiveMQ, sending 10,000 messages to the broker with
a live consumer; the producer and consumer are transactional, and the broker is configured
to use KahaDB. On failover of the broker, I see log warnings like this: 
> {code}
>  WARN | Duplicate message add attempt rejected. Message id: ID:Ade's-54024-1277715585703-0:29:1:1:240
>  WARN | Duplicate message add attempt rejected. Message id: ID:Ade's-54024-1277715585703-0:24:1:1:241
>  WARN | Duplicate message add attempt rejected. Message id: ID:Ade's-54024-1277715585703-0:23:1:1:242
>  WARN | Duplicate message add attempt rejected. Message id: ID:Ade's-54024-1277715585703-0:0:1:1:237
> {code}
> When the test has run its course, I discover that there are no messages lost - all my
messages have gone through successfully. However, I notice that there are in fact a number
of 'zombie' messages on the queue: by 'zombie' I mean that the messages are in the store,
and the JMX queueSize says they're there. However, if you browse the queue you don't see them,
and, if you have a consumer on the queue, you won't receive them either. 
> Looking into the code, in {{org.apache.activemq.store.kahadb.MessageDatabase}} you can
see the following very interesting comments concerning the log warning:
> {code}
>                // If the message ID as indexed, then the broker asked us to store
a DUP
>                 // message.  Bad BOY!  Don't do it, and log a warning.
>                 LOG.warn("Duplicate message add attempt rejected. Message id:
"+command.getMessageId());
>                 // TODO: consider just rolling back the tx.
>                 sd.messageIdIndex.put(tx, command.getMessageId(), previous);
> {code} 
> So. It seems to me that while we're detecting the condition of the broker trying to reinsert
a message that's already in the KahaDB correctly, we're not handling this case very well.
The outcome is that while we do not get any message loss, we do get worrying warning messages
*and* we get erroneous reporting of the queueSize in JMX which will mislead and create the
perception of undelivered messages.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message