activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roelof Naude <>
Subject Re: Virtual Topic and missed messages gone missing with selectorAware=true
Date Tue, 02 Nov 2010 13:51:38 GMT
hi maciej,

that sounds interesting. would you mind sharing the filter and changes 
to SelectorAwareVirtualTopicInterceptor?

On 11/02/2010 03:48 PM, Maciej Rakowicz wrote:
> Ok, i do have a somewhat contrived workaround. Basically, I decided to
> cache selector string of any queue subscription (well vtopic queue)
> that I intercept (BrokerFilter). Then I have modified
> org
> .apache
> so that if there is no subscription for a destination (here we talk
> about queue rather than vtopic again) it tries looking it up in the
> cache - well, just the selector string as it's all I need, plus due to
> the way cache is distributed it has to be easily serializable. It
> compiles the selector and tries to match it against
> MessageEvaluationContext. Of course this works with several
> assumptions in mind. First, I need this consumer to be connected at
> least once (that's when I can cache its selector). Secondly, selector
> <->  queue is 1-to-1 relation. That's perfectly acceptable in my setup.
> m.
> On 1 Nov 2010, at 12:16, Sven Vintges wrote:
>> Perhaps a work-around would be to create a garbage-service. Ie. a
>> services that pops off all messages not needed by the consumer to /
>> dev/null? Though you probably don't know which messages are not
>> needed by the subscriber since you don't know which subscribers you
>> will have... Perhaps you could create an algorithm that does
>> something like "if the consumer is up and the message is older than
>> xxx time" pop it off?
>> On Mon, 1 Nov 2010 10:25:40 +0000, "Maciej Rakowicz"<
>>> wrote:
>>> in case of selectorAware=true and consumer down, what happens to
>>> those  messages? are they discarded? Is there anything sent to one of
>>> the  advisories? I can't seem to find it anywhere.
>>> On 1 Nov 2010, at 09:48, Maciej Rakowicz wrote:
>>>> I guess, only if we keep downtimes short enough so 'matched'
>>>> messages don't expire and big enough maxPageSize or something to
>>>> keep matched messages consumed.
>>>> Does anyone know if this (i.e. AMQ-3004) is going to be worked on
>>>> in the near future?
>>>> m.
>>>> On 1 Nov 2010, at 09:32, Gary Tully wrote:
>>>>> Message expiry may help in the short term, set a time to live on
>>>>> the
>>>>> messages such that the built up unmatched messages expire and are
>>>>> removed.
>>>>> On 1 November 2010 09:20, Maciej Rakowicz
>>>>> <>   wrote:
>>>>>> All,
>>>>>> amq 5.4.0. jdk 1.6
>>>>>> Scenario:
>>>>>> broker persistence=true, selectorAware=false
>>>>>> Virtual Topic with say one queue. There is a consumer A
>>>>>> connected  to that
>>>>>> queue with a selector. All works fine. Consumer A dies,
>>>>>> messages  pile up,
>>>>>> consumer A starts back up, missed messages are redelivered. You
>>>>>> can easily
>>>>>> verify that queue receives all posted messages while consumer's
>>>>>> down.
>>>>>> Now, adding consumer B, since selectorAware=false and consumer
>>>>>> B  uses an
>>>>>> exclusive selector messages sent to consumer A are not consumed
>>>>>> by consumer
>>>>>> B. All good save the fact that all unmatched messages end up
>>>>>> polluting
>>>>>> consumer's B queue. All according to the documentation.
>>>>>> Flipping selectorAware to true solves one problem but
>>>>>> introduces  another
>>>>>> (way more important in my setup). Unmatched messages won't pile
>>>>>> up on
>>>>>> consumer's B queue which is fine. However, if consumer A dies
>>>>>> they are not
>>>>>> sent to consumer's A queue (disappearing after reaching the
>>>>>> topic  - I dunno
>>>>>> where they go) hence disabling any missed message delivery on
>>>>>> consumer's A
>>>>>> startup.
>>>>>> I there any way around it? I really need to handle all missed
>>>>>> messages with
>>>>>> no exception. At the same time I cannot allow stacking up
>>>>>> messages in each
>>>>>> and every queue although they are not matched.
>>>>>> Btw. I do not have a list of consumers in advance, the
>>>>>> subscription is fully
>>>>>> dynamic, although since broker runs persistence, once subscribed
>>>>>> I do have
>>>>>> an idea of who's subscribed even if connection is currently down.
>>>>>> I would appreciate any hints.
>>>>>> mac
>>>>> --

View raw message