activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Bain <tb...@alumni.duke.edu>
Subject Re: Messages are not discarded from queue when they're expired.
Date Fri, 28 Jul 2017 05:11:36 GMT
On Fri, Jul 21, 2017 at 8:30 AM, jbar <jan3k24@gmail.com> wrote:

> >When you say the messages "are indeed purged," do you mean to say (in more
> >words, to make sure we're clear) "When I reconnect my consumer that has
> >been offline for longer than the TTL I set, all messages older than the
> TTL
> >are expired (as confirmed by an increase in the ExpiredCount JMX attribute
> >on the queue in question or by some similar mechanism)"?
>
> In case of my project, the consumer is a big message collecting system with
> gui, so I literally see that there are no new messages after reconnecting
> it
> (even though they were sent by the producer and enqueued before) in both
> cases, with and without given TTL. As soon as the consumer goes online, the
> pending messages counter starts decreasing, and the number of dequeued
> messages starts going up.
>

I would expect the ExpiredCount JMX counter to increase, not the
DequeueCount one. That implies that the messages are in fact being sent to
the consumer even though they're expired. If so, you'd see that in the JMX
stats (the InFlightCount would be non-zero). Do you see evidence that the
messages are being dispatched to the consumer in the case of a consumer
that was offline and only reconnects after all messages are expired?


> >What about your test procedure causes them to be clearly pending for
> >more than half a second? Are you saying you're publishing messages far
> >faster than your consumer can process them and therefore there's a large
> >backlog?
>
> Yes.
>
> >How are you determining that message expiration isn't happening?
>
> I just see 10k delivered messages on a consumer side.
>

Does "delivered" mean "successfully consumed by your application," or
something else? I want to make sure I'm not misinterpreting what you're
saying.


> >With this scenario, message expiration could be taking place on either the
> >broker or the consumer
>
> With my scenario messages are pending for more than 500 ms (it takes a
> dozen
> or so seconds to dequeue them) therefore I guess it should take place on
> broker.
>

I agree with that analysis, based on that information. While messages are
still being produced, it could be taking place in both locations (though
only if the consumer finished consuming the first message while messages
are still being produced), but 500 ms after the last message is produced,
expiration should happen in the broker for all messages except the ones in
the consumer's prefetch buffer. Since your prefetch is not set for this
topic, it will use the default value of 1000 per consumer, so I'd expect to
see ~1000 messages expired on the consumer side and ~9000 expired on the
broker.

BTW, your process is only using a single consumer on this queue, right?
You're not doing anything like running 10 consumers in parallel, or running
a threadpool of 10,000 threads that each process a single message at a
time, such that all of the messages could dispatched before they expire,
are you? I assume not, but just need to ask.


> >But I would expect that 1/2
> >second after your last message is produced, your consumer would receive no
> >further messages to process.
>
> That's excactly what I'm trying to achieve.
>
> >are the clocks synchronized on whatever machines are
> >running your producers, consumers, and brokers?
>
> In my tests the producer, consumer and broker are running on the same
> machine, therefore that's not the case since they're using the same system
> time.
>

OK, thanks for confirming.


> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/ActiveMQ-Messages-are-not-discarded-from-queue-
> when-they-re-expired-tp4728731p4728770.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Can you please use the queue browser page in the web console to view the
headers of a message *before it expires* so we can confirm that the
JMSExpiration header is in fact being set (and set to a correct value
that's greater than the creation time by the TTL value)? For this to work,
you might have to extend the TTL window, because with only half a second,
your odds of accessing the message via the web console are pretty slim. I
assume that you'll find that the value of that header is correct (meaning
that the problem is in the broker and/or the consumer rather than the
producer), but let's confirm that assumption so we can rule out part of the
potential problem space.

Tim

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message