activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Bish <tabish...@gmail.com>
Subject Re: ActiveMQ JMS consumer unduly checking msg expiration and ignoring msgs? (issue with consumer clock ahead of broker clock NOT solvable via TimeStampingBrokerPlugin)
Date Wed, 22 Oct 2014 14:30:07 GMT
On 10/22/2014 10:06 AM, Gary Tully wrote:
> i think you are making a strong case for having an option to disable the
> expiry check client side. Please raise an jira issue to track it and if you
> have a moment to make a test and patch even better.
> something like jms.consumerExiryCheck=false (and leave the default as is
> (true)

The ActiveMQ .NET client has this option to solve similar complaints, so 
you can look there for inspiration.

>
> On 22 October 2014 12:04, Fred Moore <fred.moore77@gmail.com> wrote:
>
>> I see your point about the need to have fully synchronized clocks and the
>> inherent value of being strictly compliant with JMS specs... ...on the
>> other hand I have some real world scenarios at hand that drive me to be
>> extremely pragmatic:
>>
>> 1\ More often that not clocks are reasonably but not fully in synch ...we
>> need to live with it
>>
>> 2\ Message expiration is an incredibly valuable and popular feature of the
>> JMS API, and more in general of message oriented middleware ...we don't
>> want to renounce to it
>>
>> 3\ ActiveMQ provides pragmatic tools to fix clock problems between the
>> producers and the broker via the TimeStampingBrokerPl​ugin, which BTW
>> happens to cause a breach of the JMS specs ("since the timestamp that the
>> producer sees on the messages after as send() will be different from the
>> timestamp the consumer will observe when he receives the message")
>>
>> 4\ Some APIs, like ActiveMQ Stomp API, are proved to be well behaved on the
>> consumer side even in case of out-of-synch clocks
>>
>> 5\ ActiveMQ JMS API -- the best and most widely used API -- appears not to
>> have ways to inhibit the check client side check on JMSExpiration
>>
>> Any thought?
>> Is there a way to solve the JMS consumer side issue with out-of-synch
>> clocks?
>>
>> Cheers,
>> F.
>>
>>
>> On Tue, Oct 21, 2014 at 7:51 PM, Robbie Gemmell <robbie.gemmell@gmail.com>
>> wrote:
>>
>>> Hi Fred,
>>>
>>> If you are using message expiration, you ultimately need to have a handle
>>> on the degree of time synchronization between your hosts.
>>>
>>> The JMS API says for Message#getJMSExpiration() that "Clients should not
>>> receive messages that have expired; however, the JMS API does not
>> guarantee
>>> that this will not happen". The escape clause there is effectively to
>> cover
>>> the race that occurs when you decide 'this hasn't expired', only for the
>>> expiration point to have passed by the time the delivery is completed and
>>> an application checks the value. The earlier bit is saying if
>> JMSExpiration
>>>> currentTime, client applications shouldnt see the message.
>>> If a JMS client receives a message from a broker and possibly examines
>> its
>>> JMSExpiration value, it may determine that according to its local time
>> the
>>> message expiry time has already been passed, at which point it becomes an
>>> interesting question as to whether the message should be given to a
>>> consuming application or not. Some implementations will do so simply
>>> because they dont check this, and others wont because they are trying to
>>> enforce what the spec says above and according to the information at hand
>>> the message really has actually expired. Different people might prefer
>> one
>>> or the other behaviour.
>>>
>>> The same issue could reasonably exist between a producer and a broker it
>>> sends to on another host, which might never give a message out to a
>>> consumer because it had 'already expired when it arrived' for similar
>>> reason.
>>>
>>> Going back to the start this is basically to say, if you are using TTL
>>> based expiration you need to ensure an appropriate level of clock
>>> synchronization for your particular needs.
>>>
>>> I'll leave the specifics of the questions to someone familiar with the
>>> code.
>>>
>>> Robbie
>>>
>>> On 21 October 2014 16:14, Fred Moore <fred.moore77@gmail.com> wrote:
>>>
>>>> Hi folks,
>>>>
>>>> I found a situation where the handling of timestamps/expirations by
>>>> ActiveMQ 5.10 appears not to be correct, and unfortunately -- being
>> this
>>> a
>>>> consumer side issue -- it does not appear to be solvable via classic
>>>> TimeStampingBrokerPlugin tweaks.
>>>>
>>>> Scenario:
>>>>
>>>> S1\ Broker B1 runs on HOST1
>>>> S2\ Producer P1 is a java JMS application running on HOST1
>>>> S3\ HOST1 is on UTC timezone
>>>> S4\ B1 has BrokerTimestampingPlugin with futureOnly=true configured
>>>>      (even if I think it does not play any role in this specific case)
>>>> S4\ Consumer C1 is a java JMS API application running on HOST2
>>>> S5\ Consumer C2 is a java STOMP API application running on HOST2
>>>> S6\ HOST2 is on UTC timezone and its clock is 3 mins ahead of HOST1
>> clock
>>>>
>>>> Test case:
>>>>
>>>> T1\ P1 sends  message M1 with TTL=1 minute
>>>> T2\ C1 on HOST2 immediately tries to receive a message but it does not
>>> find
>>>> it because
>>>>      according to HOST2 clock that message has already expired (see [*]
>>>> trace)
>>>> T3\ C2 on HOST2 tries to receive a message and it succeeds
>>>>
>>>> Questions/congeptures:
>>>>
>>>> Q1\ The behaviour looks client API/technology dependent, is this
>>> expected?
>>>>     [I think it's not]
>>>> Q2\ It looks like the JMS layer in C1 is re-checking expiration on
>>>> candidate messages
>>>>     "served" to it by the broker, causing the reported issue: is there a
>>> way
>>>> to inhibit
>>>>     this extra logic in JMS?
>>>> Q3\ Is my assumption about BrokerTimestampingPlugin being useless here
>>>> correct?
>>>>      [It can only influence the produced messages and not the consumed
>>> ones]
>>>> If there is no general solution to this it means that messages are
>>> wrongly
>>>> ignored by ActiveMQ JMS consumers every time these [not so uncommon]
>>>> conditions are verified:
>>>>
>>>> F1\ the consumer clock runs ahead of the broker clock by X seconds
>>>> F2\ the TTL of messages is less than X seconds
>>>> F3\ the consumer is a java JMS application
>>>>
>>>> Please help!
>>>> Cheers,
>>>> F.
>>>>
>>>>
>>>> -----
>>>> [*] Trace showing C1 misperception of M1 expiration
>>>>
>>>> 2014-10-21 15:53:11,987 [TestListener] DEBUG ActiveMQMessageConsumer
>>>> .dequeue - ID:gd-57426-1413899591527-1:1:1:1 received expired message:
>>>> MessageDispatch {commandId = 0, responseRequired = false, consumerId =
>>>> ID:gd-57426-1413899591527-1:1:1:1, destination = queue://TEST, message
>> =
>>>> ActiveMQTextMessage {commandId = 9, responseRequired = false,
>> messageId =
>>>> ID:narsil104.narsil104-29953-1413889401564-3:3:1:1:5,
>>> originalDestination =
>>>> null, originalTransactionId = null, producerId =
>>>> ID:narsil104.narsil104-29953-1413889401564-3:3:1:1, destination =
>>>> queue://TEST, transactionId = null, expiration = 1413898985466,
>>> timestamp =
>>>> 1413898385466, arrival = 0, brokerInTime = 1413898385466,
>> brokerOutTime =
>>>> 1413898389795, correlationId = , replyTo = null, persistent = false,
>>> type =
>>>> , priority = 0, groupID = null, groupSequence = 0, targetConsumerId =
>>> null,
>>>> compressed = false, userID = null, content =
>>>> org.apache.activemq.util.ByteSequence@642a590d, marshalledProperties =
>>>> null, dataStructure = null, redeliveryCounter = 0, size = 0,
>> properties =
>>>> null, readOnlyProperties = true, readOnlyBody = true, droppable =
>> false,
>>>> jmsXGroupFirstForConsumer = false, text = Exp 5 min}, redeliveryCounter
>>> = 0


-- 
Tim Bish
Sr Software Engineer | RedHat Inc.
tim.bish@redhat.com | www.redhat.com
skype: tabish121 | twitter: @tabish121
blog: http://timbish.blogspot.com/


Mime
View raw message