activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alec Henninger <alechennin...@gmail.com>
Subject Re: Alternative SubQueueSelectorCacheBroker implementations (for replicating the cache)
Date Sat, 08 Sep 2018 21:54:15 GMT
I did a little experimentation and what seems to make the most sense is
keep the broker plugin as a concrete class – there is necessary logic there
that is part of activemq's configuration interface so it might not make
sense to have alternate implementations. The persistence interactions can
be pulled out to an interface okay enough it seems. I'll try and get a
patch together for closer review.

On Sat, Sep 8, 2018 at 1:40 PM Alec Henninger <alechenninger@gmail.com>
wrote:

> Hi activemq devs,
>
> Background: We're running a multitenant activemq pair using selector aware
> virtual topics extensively but also need truly durable subscriptions.
> The SubQueueSelectorCacheBroker plugin was developed for this purpose as we
> understand, however it persists the selector cache in a File, and we'd like
> to instead use a shared/replicated cache that doesn't depend on a node
> first storing the cached consumer selector locally first. The reason for
> this is, with the current implementation, there is an edge case where if
> both:
>
> 1. A active broker has not yet cached a consumer's selector (e.g.
> secondary broker becomes primary that hasn't yet received connection from
> said consumer)
> 2. Producer connects and starts publishing messages before consumer
>
> ...then those messages will be lost. In some domains, any message loss is
> really undesirable so we want to do everything we can to prevent that while
> still using selector aware virtual topics. We'd just turn off
> selectorAware, but then we have to deal with message build up for consumers
> using selectors, and we have little control over how/when consumers use
> selectors.
>
> So given those constraints, we'd like to contribute an alternative
> implementation of the plugin that allows use of something other than a
> File, but before we get too far on that we'd like to get the opinion of
> activemq devs on how to approach that. I can talk about a few options I've
> thought of so far, and of course if you have others, would love to hear
> them.
>
> Option 1: Change SubQueueSelectorCacheBroker to an interface.
> Right now, SelectorAwareVirtualTopicInterceptor is tightly coupled to the
> concrete SubQueueSelectorCacheBroker type. This makes extending this plugin
> (at least with current implementation) difficult. If the interceptor was
> only tied to an interface, then of course this enables alternative
> implementations. The interface only needs a single `Set<String>
> getSelector(String destination)` method.
>
> Option 2: (Not mutually exclusive with 1) Inject abstract persistence
> mechanism into SubQueueSelectorCacheBroker instead of injecting File
> This removes most of the impact of coupling mentioned in Option 1, since
> the plugin itself would be decoupled from a specific persistence mechanism.
> Not sure what the right abstraction is here. Could inject an interface as
> low-level as one method for `persistCache(ConcurrentMap<String,
> Set<String>> subSelectorCache)` and another for `ConcurrentMap<String,
> Set<String>> getCache()`. In this case, the periodic flushing mechanism
> still exists in the plugin, it just calls out to the this interface do the
> initial read and periodic flushes. Let's call this option 2.1. This seems
> like a missed opportunity to also abstract out how the cache is updated–if
> we're using a database or key:value store, we might use some other
> algorithm to update the nonvolatile cache than periodic flushing. So
> perhaps instead an interface more like the selector map itself (put/get
> methods). Let's call that option 2.2. It's tempting to also pull out the in
> memory caching as responsibility of this interface as well, but these feels
> so ubiquitously useful it might make sense to leave that inside the plugin.
> Let's call moving the in memory caching pieces inside the interface
> implementation option 2.3.
>
> Hopefully that made sense. Any thoughts would be greatly appreciated!
>
> Thanks,
> Alec
>
> PS Any Red Hatters feel free to contact me at ahenning@redhat.com if
> you'd like to chat in person
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message