activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From magellings <>
Subject Re: duplicates on master/slave broker failover
Date Wed, 30 Sep 2009 12:29:27 GMT

For the developers of NMS:

There’s a mediation pattern we could use to handle duplicate messages.  It
is the idempotent consumer pattern [1].  I believe it's a justifiable
approach to handling situations where the broker and its consumer(s) cannot
resolve the problem of duplicate messaging on their own.

It is already a justifiable approach with java clients, as a solution to
this already exists.  If we were writing our consumers in java we actually
could use Apache Camel [2,3].  

I don't believe this functionality exists in NMS beyond a preexisting audit
layer.  Our company needs to provide the capability of ignoring duplicate
messages across clustered consumers and if restarts of the consumer occurs.

So unless anyone knows that this functionality does exist already (please
speak up), we have decided we'll be adding an extra audit layer to our
consumers to recognize duplicate messages using the idempotent consumer
pattern.  We'll be specifically focusing on a persistent based repository to
store messages (not in memory at this point).  

I think it would make sense implementing this into the NMS framework for
everyone else to take advantage of (which is why I'd like NMS developer
feedback on this). I've coded the async connect patch for the failover
protocol which was recently applied to NMS 1.2 [4].  If this makes sense to
the developers of NMS then let me know.  I will create an item in the issue
tracker for version 1.2, attempt to code this into the NMS ActiveMQ provider
and submit a patch for review.


magellings wrote:
> Evaluating this further it appears the broker is not persisting any
> information back to the database when it dispatches messages to consumers. 
> I verified by removing update privs on the ACTIVEMQ_MSGS table as well as
> insert, update, and delete privs on the ACTIVEMQ_ACKS table.  Yes we are
> using JDBC Master/Slave.  It seems that if this is the case the master
> broker must be tracking dispatches of messages to consumers in memory.  So
> it would be impossible for a new master broker to know if a consumer had
> received a message.  Does this seem correct to anyone more knowledgeable
> on how JDBC Master/Slave works?  Also, what is the ACTIVEMQ_ACKS table
> for?  It almost seems like this table would be used to keep track of the
> acks a consumer sent back to a broker or something and possible would be
> used to resolve the possiblity of a duplicate message being dispatched to
> another consumer when a slave became master.  Except like I said after
> revoking insert, delete, update privs on this table the broker continues
> to work just fine.  So it doesn't look like the table is utilized.  
> Any help is appreciated.  Thanks

View this message in context:
Sent from the ActiveMQ - User mailing list archive at

View raw message