activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Tully <gary.tu...@gmail.com>
Subject Re: ActiveMQ JMS consumer unduly checking msg expiration and ignoring msgs? (issue with consumer clock ahead of broker clock NOT solvable via TimeStampingBrokerPlugin)
Date Wed, 22 Oct 2014 14:06:00 GMT
i think you are making a strong case for having an option to disable the
expiry check client side. Please raise an jira issue to track it and if you
have a moment to make a test and patch even better.
something like jms.consumerExiryCheck=false (and leave the default as is
(true)

On 22 October 2014 12:04, Fred Moore <fred.moore77@gmail.com> wrote:

> I see your point about the need to have fully synchronized clocks and the
> inherent value of being strictly compliant with JMS specs... ...on the
> other hand I have some real world scenarios at hand that drive me to be
> extremely pragmatic:
>
> 1\ More often that not clocks are reasonably but not fully in synch ...we
> need to live with it
>
> 2\ Message expiration is an incredibly valuable and popular feature of the
> JMS API, and more in general of message oriented middleware ...we don't
> want to renounce to it
>
> 3\ ActiveMQ provides pragmatic tools to fix clock problems between the
> producers and the broker via the TimeStampingBrokerPl​ugin, which BTW
> happens to cause a breach of the JMS specs ("since the timestamp that the
> producer sees on the messages after as send() will be different from the
> timestamp the consumer will observe when he receives the message")
>
> 4\ Some APIs, like ActiveMQ Stomp API, are proved to be well behaved on the
> consumer side even in case of out-of-synch clocks
>
> 5\ ActiveMQ JMS API -- the best and most widely used API -- appears not to
> have ways to inhibit the check client side check on JMSExpiration
>
> Any thought?
> Is there a way to solve the JMS consumer side issue with out-of-synch
> clocks?
>
> Cheers,
> F.
>
>
> On Tue, Oct 21, 2014 at 7:51 PM, Robbie Gemmell <robbie.gemmell@gmail.com>
> wrote:
>
> > Hi Fred,
> >
> > If you are using message expiration, you ultimately need to have a handle
> > on the degree of time synchronization between your hosts.
> >
> > The JMS API says for Message#getJMSExpiration() that "Clients should not
> > receive messages that have expired; however, the JMS API does not
> guarantee
> > that this will not happen". The escape clause there is effectively to
> cover
> > the race that occurs when you decide 'this hasn't expired', only for the
> > expiration point to have passed by the time the delivery is completed and
> > an application checks the value. The earlier bit is saying if
> JMSExpiration
> > > currentTime, client applications shouldnt see the message.
> >
> > If a JMS client receives a message from a broker and possibly examines
> its
> > JMSExpiration value, it may determine that according to its local time
> the
> > message expiry time has already been passed, at which point it becomes an
> > interesting question as to whether the message should be given to a
> > consuming application or not. Some implementations will do so simply
> > because they dont check this, and others wont because they are trying to
> > enforce what the spec says above and according to the information at hand
> > the message really has actually expired. Different people might prefer
> one
> > or the other behaviour.
> >
> > The same issue could reasonably exist between a producer and a broker it
> > sends to on another host, which might never give a message out to a
> > consumer because it had 'already expired when it arrived' for similar
> > reason.
> >
> > Going back to the start this is basically to say, if you are using TTL
> > based expiration you need to ensure an appropriate level of clock
> > synchronization for your particular needs.
> >
> > I'll leave the specifics of the questions to someone familiar with the
> > code.
> >
> > Robbie
> >
> > On 21 October 2014 16:14, Fred Moore <fred.moore77@gmail.com> wrote:
> >
> > > Hi folks,
> > >
> > > I found a situation where the handling of timestamps/expirations by
> > > ActiveMQ 5.10 appears not to be correct, and unfortunately -- being
> this
> > a
> > > consumer side issue -- it does not appear to be solvable via classic
> > > TimeStampingBrokerPlugin tweaks.
> > >
> > > Scenario:
> > >
> > > S1\ Broker B1 runs on HOST1
> > > S2\ Producer P1 is a java JMS application running on HOST1
> > > S3\ HOST1 is on UTC timezone
> > > S4\ B1 has BrokerTimestampingPlugin with futureOnly=true configured
> > >     (even if I think it does not play any role in this specific case)
> > > S4\ Consumer C1 is a java JMS API application running on HOST2
> > > S5\ Consumer C2 is a java STOMP API application running on HOST2
> > > S6\ HOST2 is on UTC timezone and its clock is 3 mins ahead of HOST1
> clock
> > >
> > >
> > > Test case:
> > >
> > > T1\ P1 sends  message M1 with TTL=1 minute
> > > T2\ C1 on HOST2 immediately tries to receive a message but it does not
> > find
> > > it because
> > >     according to HOST2 clock that message has already expired (see [*]
> > > trace)
> > > T3\ C2 on HOST2 tries to receive a message and it succeeds
> > >
> > > Questions/congeptures:
> > >
> > > Q1\ The behaviour looks client API/technology dependent, is this
> > expected?
> > >    [I think it's not]
> > > Q2\ It looks like the JMS layer in C1 is re-checking expiration on
> > > candidate messages
> > >    "served" to it by the broker, causing the reported issue: is there a
> > way
> > > to inhibit
> > >    this extra logic in JMS?
> > > Q3\ Is my assumption about BrokerTimestampingPlugin being useless here
> > > correct?
> > >     [It can only influence the produced messages and not the consumed
> > ones]
> > >
> > > If there is no general solution to this it means that messages are
> > wrongly
> > > ignored by ActiveMQ JMS consumers every time these [not so uncommon]
> > > conditions are verified:
> > >
> > > F1\ the consumer clock runs ahead of the broker clock by X seconds
> > > F2\ the TTL of messages is less than X seconds
> > > F3\ the consumer is a java JMS application
> > >
> > > Please help!
> > > Cheers,
> > > F.
> > >
> > >
> > > -----
> > > [*] Trace showing C1 misperception of M1 expiration
> > >
> > > 2014-10-21 15:53:11,987 [TestListener] DEBUG ActiveMQMessageConsumer
> > > .dequeue - ID:gd-57426-1413899591527-1:1:1:1 received expired message:
> > > MessageDispatch {commandId = 0, responseRequired = false, consumerId =
> > > ID:gd-57426-1413899591527-1:1:1:1, destination = queue://TEST, message
> =
> > > ActiveMQTextMessage {commandId = 9, responseRequired = false,
> messageId =
> > > ID:narsil104.narsil104-29953-1413889401564-3:3:1:1:5,
> > originalDestination =
> > > null, originalTransactionId = null, producerId =
> > > ID:narsil104.narsil104-29953-1413889401564-3:3:1:1, destination =
> > > queue://TEST, transactionId = null, expiration = 1413898985466,
> > timestamp =
> > > 1413898385466, arrival = 0, brokerInTime = 1413898385466,
> brokerOutTime =
> > > 1413898389795, correlationId = , replyTo = null, persistent = false,
> > type =
> > > , priority = 0, groupID = null, groupSequence = 0, targetConsumerId =
> > null,
> > > compressed = false, userID = null, content =
> > > org.apache.activemq.util.ByteSequence@642a590d, marshalledProperties =
> > > null, dataStructure = null, redeliveryCounter = 0, size = 0,
> properties =
> > > null, readOnlyProperties = true, readOnlyBody = true, droppable =
> false,
> > > jmsXGroupFirstForConsumer = false, text = Exp 5 min}, redeliveryCounter
> > = 0
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message