activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jgenender <jgenen...@apache.org>
Subject Re: ActiveMQ and Artemis reliability - Messages lost
Date Tue, 13 Nov 2018 02:36:47 GMT
I looked at the jmstool, and I am not convinced of any bugs in Artemis and
AMQ.  It uses an external logging mechanism to consider the state of
messages, which is likely not what will be in ActiveMQ or Artemis, nor may
it be truly representative of messages that have been passed (or not).  What
does the tool do from a log analysis is the consumption dies due to a
disconnect and reconnects? 

If you want to test this, then you need consumers/producers with counts and
at the end, verify the enqueue/dequeue counts and message states WITHIN the
brokers.

I further believe that the transactional logging mechanisms within the
brokers themselves mostly guarantee little loss.  Especially if XA is being
used.  In fact, if you restart the broker (at least for AMQ), and you hit
the PREPARE phase of the XA, those will be in a PREPARED state until you
decide to do something with them, of which is not a "lost message" in any
sense.  By spec you need to either RECOVER that message, heuristically
decide on its disposition, or it remains in PREPARED until you do something
with it.   With regard to jmstool, it would likely flag those as
gone/missing. 

I'm in full agreement with jbertram on this.  You need to simplify your
tests and verify the location of the messages.   If you are killing the
broker in the manner that you are, you are more likely to get partial writes
(assuming the broker's shutdown hook doesn't fire).  Your broker is only as
good as the writes to the disk are verified and completed.

For AMQ and Artemis, my bets are on very little message loss if using full
persistence and you will need a more robust way of testing if messages have
been produced/consumed/lost.  The tool that you are using doesn't quite cut
it due to the LogAnalyzer which likely wasn't made to handle
disconnects/failover/re-reads.   It lacks serious error handling within its
logging to be more accurate on whether messages are truly produced and
consumed.  I'm sure it works mostly good on a stable broker to measure
throughput, etc, but if you are going to create broken connections, it needs
a lot more code to handle that for accuracy of where the messages are and
whether there really are dups or other negative events.

Just my .02.



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Mime
View raw message