activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ttmdev <joe.fernan...@ttmsolutions.com>
Subject Re: Consuming persistent messages from a non-durable subscriber
Date Fri, 28 Mar 2008 17:32:38 GMT

I think I would be a bit more inclined to look into virtual topics. Primarily
because they're easier to work with - vs durable subscriptions - and they
also offer the functionality provided by queues.   So you get the best of
both worlds ;) 

Joe
  

Ramit Arora wrote:
> 
> Hi Joe,
> Thanks for your time. My application requires a hub & spoke architecture
> with around 100 publishers & 500 consumers at peak load. All publishers
> would be pushing messages to different topics. One consumer would be
> durable as it needs to collect all messages. The rest would be online
> sporadically (actually they are end-user GUIs & I intend to stream
> messages in real time to them). So I intended to make them non-durable.
> What I want is that if my broker goes down, all my consumers should
> failover to the slave & not loose any messages. I am using NFS based
> master/slave. I am observing that my non-durable subscribers are losing
> messages on failover if they are lagging behind the producers. 
> I am concerned about the cost of using 500 durable subscribers. I'll be
> considering the use of retroactive consumers & virtual destinations. What
> would be the recommended thing to do in my case?
> And BTW, the user guide is awesome!
> 
> Thanks,
> Ramit 
> 
> ttmdev wrote:
>> 
>> Your non-durable subscriber will only get those messages that were
>> published while it is active.  When you bounce the broker it does not
>> retain the identity of your non-durable subscriber. So when the broker
>> comes back up, it doesn't consider your non-durable subscriber to have
>> been active when the persisted messages were published.    
>> 
>> Can you use durable subscribers? If not, look into retroactive consumers
>> or better yet, virtual topics. With the latter you get the functionality
>> of both a topic and queue and you don't have to worry about setting up
>> unique identifiers as you would have to when using durable subscriptions.
>> 
>> http://activemq.apache.org/virtual-destinations.html
>> 
>> http://activemq.apache.org/retroactive-consumer.html
>> 
>> Joe
>> Goto www.ttmsolutions.com for a free ActiveMQ user guide
>> 
>> 
>> 
>> Ramit Arora wrote:
>>> 
>>> Hi All,
>>> I have a situation where I have to consume persisent messages from many
>>> non-durable subscribers. My publishers push persistent messages to a
>>> topic, which are consumed asynchronously by non-durable subscribers. I
>>> am using the AMQ Persistence Adapter & CLIENT_ACKNOWLEDGE. If I kill a
>>> non-durable subscriber & bring it up again, some messages are lost,
>>> which is expected. But consider the following case:
>>> 1. Start a publisher publishing persistent messages to a topic.
>>> 2. Start a non-durable subscriber which is slow compared to the
>>> publisher.
>>> 3. Kill the broker when 100 messages have been published but only 50
>>> have been consumed.
>>> 4. Since the publisher & subscriber are failover clients, they wait for
>>> the broker to come up again.
>>> 5. Restart the broker.
>>> 6. Try to get the remaining 50 messages to the non-durable subscriber.
>>> 
>>> Step 6 does not work in both 5.0 & the latest 5.1 snapshot. Shouldn't
>>> the remaining 50 messages be delivered? The JMS spec says:
>>> "A JMS provider must deliver a PERSISTENT message once-and-only-once.
>>> This
>>> means a JMS provider failure must not cause it to be lost, and it must
>>> not
>>> deliver it twice. PERSISTENT (once-and-only-once) and NON_PERSISTENT
>>> (at-most-once)
>>> message delivery are a way for a JMS client to select between delivery
>>> techniques that may lose a messages if a JMS provider dies and those
>>> which
>>> take extra effort to insure that messages can survive such a failure.
>>> There is
>>> typically a performance/reliability trade-off implied by this choice.
>>> When a
>>> client selects the NON_PERSISTENT delivery mode, it is indicating that
>>> it
>>> values performance over reliability; a selection of PERSISTENT reverses
>>> the
>>> requested trade-off."
>>> 
>>> Queues donot suffer from this problem. Please let me know if I am
>>> missing something.
>>> 
>>> Thanks,
>>> Ramit
>>> 
>>> 
>>> 
>> 
>> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Consuming-persistent-messages-from-a-non-durable-subscriber-tp16348721s2354p16356200.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message