activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mtaylor <...@git.apache.org>
Subject [GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support
Date Fri, 04 Nov 2016 15:35:20 GMT
Github user mtaylor commented on the issue:

    https://github.com/apache/activemq-artemis/pull/875
  
    Firstly we are making some major improvements to the Artemis addressing model that might
help here.
    
    @mattrpav 
    
    >  selectorAware: This allows an interested subscriber to only receive messages in
their queue that match a given selector. This broker-side pre-filtering is an optimization
which reduces the on-disk requirements for really busy Virtual 
    
    This is how normal Artemis subscriptions currently work.  The Artemis addressing/routing
model is different from ActiveMQ.  Messages are routed straight to subscription queues, and
filtering happens on the broker at routing (send) time.
    
    > 2) transactedSend: This is a (imho & experience) critical reliability requirement.
In use cases where reliability is a critical requirement: when a producer sends a message
to the broker, even during non-transacted sessions, the broker should transact the dispatch
to the consumer queues before ack'n the producer. This ensures no messages are lost in the
event of unplanned outage while a broker is dispatching messages to the consumer queues.
    
    We also already have this behaviour, but it works slightly differently.  We don't create
a transaction that puts the message on all consumer queues then returns an ack.  Instead,
we persist the message and return an ack (depending on the protocol behaviour).  In the event
of the broker going down just before the message is persisted, but before routing, it will
retry routing on restart.  Which gives you the same behaviour.
    
    > 3) concurrentSend: This is a critical performance enhancement for Virtual Topics
where there are a large number of consumer queues. The behavior is that the broker processes
queue dispatch in parallel to consumer queues in order to improve throughput and lower latency.
    
    I don't think I fully follow what you are saying here, but, in Artemis there's only consumer
queues.  We route messages direct to consumer queues.  All consumption of messages form those
queues and dispatching to the consumer is done in parrallel using a shared thread pool.
    
    > iii. While outside the scope of this PR, I thought I'd mention it here as it is related--
there has long been a desire to enhance ActiveMQ to support single-message store. That is,
for all subscriptions, only store a single copy of a message. This becomes really apparent
in use cases with a large number of Virtual Consumers.
    
    We already do this.  A message is only persisted once, we then use reference counting
to delete it once it's been consumed from all queues.
    
    



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message