activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alec Henninger <alechennin...@gmail.com>
Subject Re: Alternative SubQueueSelectorCacheBroker implementations (for replicating the cache)
Date Tue, 18 Sep 2018 14:10:59 GMT
Hi devs, I opened a pull request for this:
https://github.com/apache/activemq/pull/297

Review would be greatly appreciated!

Cheers,
Alec

On Sat, Sep 8, 2018 at 5:54 PM Alec Henninger <alechenninger@gmail.com>
wrote:

> I did a little experimentation and what seems to make the most sense is
> keep the broker plugin as a concrete class – there is necessary logic there
> that is part of activemq's configuration interface so it might not make
> sense to have alternate implementations. The persistence interactions can
> be pulled out to an interface okay enough it seems. I'll try and get a
> patch together for closer review.
>
> On Sat, Sep 8, 2018 at 1:40 PM Alec Henninger <alechenninger@gmail.com>
> wrote:
>
>> Hi activemq devs,
>>
>> Background: We're running a multitenant activemq pair using selector
>> aware virtual topics extensively but also need truly durable subscriptions.
>> The SubQueueSelectorCacheBroker plugin was developed for this purpose as we
>> understand, however it persists the selector cache in a File, and we'd like
>> to instead use a shared/replicated cache that doesn't depend on a node
>> first storing the cached consumer selector locally first. The reason for
>> this is, with the current implementation, there is an edge case where if
>> both:
>>
>> 1. A active broker has not yet cached a consumer's selector (e.g.
>> secondary broker becomes primary that hasn't yet received connection from
>> said consumer)
>> 2. Producer connects and starts publishing messages before consumer
>>
>> ...then those messages will be lost. In some domains, any message loss is
>> really undesirable so we want to do everything we can to prevent that while
>> still using selector aware virtual topics. We'd just turn off
>> selectorAware, but then we have to deal with message build up for consumers
>> using selectors, and we have little control over how/when consumers use
>> selectors.
>>
>> So given those constraints, we'd like to contribute an alternative
>> implementation of the plugin that allows use of something other than a
>> File, but before we get too far on that we'd like to get the opinion of
>> activemq devs on how to approach that. I can talk about a few options I've
>> thought of so far, and of course if you have others, would love to hear
>> them.
>>
>> Option 1: Change SubQueueSelectorCacheBroker to an interface.
>> Right now, SelectorAwareVirtualTopicInterceptor is tightly coupled to the
>> concrete SubQueueSelectorCacheBroker type. This makes extending this plugin
>> (at least with current implementation) difficult. If the interceptor was
>> only tied to an interface, then of course this enables alternative
>> implementations. The interface only needs a single `Set<String>
>> getSelector(String destination)` method.
>>
>> Option 2: (Not mutually exclusive with 1) Inject abstract persistence
>> mechanism into SubQueueSelectorCacheBroker instead of injecting File
>> This removes most of the impact of coupling mentioned in Option 1, since
>> the plugin itself would be decoupled from a specific persistence mechanism.
>> Not sure what the right abstraction is here. Could inject an interface as
>> low-level as one method for `persistCache(ConcurrentMap<String,
>> Set<String>> subSelectorCache)` and another for `ConcurrentMap<String,
>> Set<String>> getCache()`. In this case, the periodic flushing mechanism
>> still exists in the plugin, it just calls out to the this interface do the
>> initial read and periodic flushes. Let's call this option 2.1. This seems
>> like a missed opportunity to also abstract out how the cache is updated–if
>> we're using a database or key:value store, we might use some other
>> algorithm to update the nonvolatile cache than periodic flushing. So
>> perhaps instead an interface more like the selector map itself (put/get
>> methods). Let's call that option 2.2. It's tempting to also pull out the in
>> memory caching as responsibility of this interface as well, but these feels
>> so ubiquitously useful it might make sense to leave that inside the plugin.
>> Let's call moving the in memory caching pieces inside the interface
>> implementation option 2.3.
>>
>> Hopefully that made sense. Any thoughts would be greatly appreciated!
>>
>> Thanks,
>> Alec
>>
>> PS Any Red Hatters feel free to contact me at ahenning@redhat.com if
>> you'd like to chat in person
>>
>>
>>

-- 
Alec
(570) 856-2428

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message