activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Davies <rajdav...@gmail.com>
Subject Re: Issue creating a distributed queue using store and forward.
Date Thu, 08 Oct 2009 16:42:48 GMT
Hi Eric,

sorry for being so quick in my reply - in the broker xml config set  
the prefetchSize property on the network - e.g.

<networkConnectors>
         <networkConnector
              uri="static://(tcp://localhost:61617)"
              name="Connection to 61617"
              dynamicOnly="true"
	     prefetchSize="1"/>

  </networkConnectors>
On 8 Oct 2009, at 17:16, Eric Van Dewoestine wrote:

> I guess I'm not sure what setting you are referring to.  I tried
> setting a couple different prefetch policies on the tcp connector like
> so:
>
> 1st attempt:
> <transportConnector
>  uri="tcp://${broker.host}:0?jms.prefetchPolicy.queuePrefetch=1"
>  discoveryUri="multicast://default?group=${broker.group}"/>
>
> 2nd attempt:
> <transportConnector
>  uri="tcp://${broker.host}:0?jms.prefetchPolicy.all=1"
>  discoveryUri="multicast://default?group=${broker.group}"/>
>
> But neither of them seemed to change the behavior.  Are you referring
> to another setting?  Can you provide an example of the setting you are
> referring to?
>
> On Thu, Oct 08, 2009 at 04:22:11PM +0100, Rob Davies wrote:
>> do you set the prefetchSize property on the network itself ? Setting
>> it on the consumers wouldn't make any difference
>> On 8 Oct 2009, at 15:39, Eric Van Dewoestine wrote:
>>
>>> I tried setting the prefetchSize to 1 for both consumers, but
>>> unfortunately, that didn't resolve the issue.  Messages still end up
>>> stuck on B2 when C2 is stopped.
>>>
>>> On Thu, Oct 08, 2009 at 09:33:06AM +0100, Rob Davies wrote:
>>>> set the prefetchSize=1 on your network - to avoid orphaned messages
>>>> on
>>>> B2
>>>>
>>>> On 7 Oct 2009, at 20:06, Eric Van Dewoestine wrote:
>>>>
>>>>> Hmm, that seems like a flawed approach if the broker it visited  
>>>>> has
>>>>> changed state and is now eligible to process those messages.   
>>>>> How is
>>>>> this type of fail over intended to be handled with activemq?  Is
>>>>> there
>>>>> an alternate solution that I'm missing?
>>>>>
>>>>> On Wed, Oct 07, 2009 at 11:43:49AM -0700, Joe Fernandez wrote:
>>>>>>
>>>>>> I think that by design a message will not be forwarded to a  
>>>>>> broker
>>>>>> that it
>>>>>> has already visited.
>>>>>>
>>>>>> Joe
>>>>>> http://www.ttmsolutions.com
>>>>>>
>>>>>>
>>>>>> Eric Van wrote:
>>>>>>>
>>>>>>> ActiveMQ 5.3.0_SNAPSHOT (Sep 8th according to the snapshots
>>>>>>> listing)
>>>>>>>
>>>>>>> I'm running into an issue with the store and forward feature
of
>>>>>>> activemq, which I using in an attempt to create a highly  
>>>>>>> available
>>>>>>> distributed queue.  I'm trying to figure out if the issue is
a
>>>>>>> misconfiguration on my part, expected behavior of activemq, or
a
>>>>>>> bug.
>>>>>>>
>>>>>>> The summary of the problem is that given 2 brokers, B1 and B2,
>>>>>>> which
>>>>>>> each have one consumer, C1 and C2, which are subscribed to the
>>>>>>> same
>>>>>>> queue.  If I stop a consumer on one of the brokers, the pending
>>>>>>> messages from that broker are not always forwarded to the other
>>>>>>> broker
>>>>>>> which still has a consumer, leading to those messages getting
>>>>>>> indefinitely stuck.
>>>>>>>
>>>>>>> The steps I use to reproduce this scenario are as follows (Note:
>>>>>>> all
>>>>>>> producing and consuming is performed over the stomp transport):
>>>>>>>
>>>>>>> Couple notes about the consumers:
>>>>>>> - they have a prefetchSize of 40
>>>>>>> - the processing of messages can take some time, so for the
>>>>>>> purposes
>>>>>>> of this exercise, I've created a simple consumer that sleeps
for
>>>>>>> 10
>>>>>>> seconds before sending the message ack (using client-individual
>>>>>>> ack
>>>>>>> mode)
>>>>>>>
>>>>>>> 1. start both brokers (B1 and B2). The consumers (C1 and C2)
are
>>>>>>> not
>>>>>>> yet running.
>>>>>>> 2. produce a few thousand messages to B1
>>>>>>> Note: B1 now has a few thousand pending messages and B2 has 0.
>>>>>>> 3. start consumer C2 (listing for messages from B2)
>>>>>>> Note: messages are are successfully received and begin  
>>>>>>> processing
>>>>>>> (monitoring the brokers shows pending messages decreasing). Now
>>>>>>> B2
>>>>>>> has all the pending messages and B1 has 0.
>>>>>>> 4. start consumer C1 (listing for messages from B1)
>>>>>>> Note: no messages are received, which is another issue I have
>>>>>>> since
>>>>>>> B2 now has thousands of pending messages which C1 could help
>>>>>>> process, but instead sits idle while C2 is forced to handle all
>>>>>>> the
>>>>>>> messages.
>>>>>>> 5. stop consumer C2
>>>>>>> Note: now I have thousands of messages sitting on B2 and 0 on
B1
>>>>>>> where a C1 is alive and ready to handle them.  So at this point,
>>>>>>> despite having a consumer running, thousands of messages are
>>>>>>> stuck
>>>>>>> in the queue.
>>>>>>> 6. stop consumer C1
>>>>>>> Note: now I have no consumers.  Stopping and restarting C1 has
 
>>>>>>> no
>>>>>>> effect on the pending messages sitting on B1's queue.
>>>>>>> 7. stop both brokers
>>>>>>> 8. start B1, then start B2
>>>>>>> 9. start C1
>>>>>>> Note: now all messages have migrated from B2 to B1 and C1 is
>>>>>>> again
>>>>>>> processing messages.
>>>>>>>
>>>>>>> So after step 5, the only way to recover from the stuck messages
>>>>>>> is to
>>>>>>> restart the brokers.
>>>>>>>
>>>>>>> Below is my current connector config which I have on both  
>>>>>>> brokers.
>>>>>>> I've tried playing with the various properties of the connector,
>>>>>>> but
>>>>>>> it seems as though no matter what I try the above scenario
>>>>>>> continues
>>>>>>> to occur.
>>>>>>>
>>>>>>> <networkConnector
>>>>>>>  name="default-nc"
>>>>>>>  uri="multicast://default?group=${broker.group}"
>>>>>>>  dynamicOnly="true"
>>>>>>>  networkTTL="25"
>>>>>>>  suppressDuplicateQueueSubscriptions="true"/>
>>>>>>>
>>>>>>>
>>>>>>> So, is this an activemq bug? Am I mis-using activemq? Is there
>>>>>>> some
>>>>>>> other way to achieve a highly available distributed queue?
>>>>>>>
>>>>>>> Any help in this regard is greatly appreciated.
>>>>>>>
>>>>>>> --
>>>>>>> eric
>>>>>>>
>>>>>>>
>>>>>
>>>>> --
>>>>> eric
>>>>
>>>> Rob Davies
>>>> http://twitter.com/rajdavies
>>>> I work here: http://fusesource.com
>>>> My Blog: http://rajdavies.blogspot.com/
>>>> I'm writing this: http://www.manning.com/snyder/
>>>
>>> --
>>> eric
>>
>> Rob Davies
>> http://twitter.com/rajdavies
>> I work here: http://fusesource.com
>> My Blog: http://rajdavies.blogspot.com/
>> I'm writing this: http://www.manning.com/snyder/
>>
>>
>>
>>
>>
>
> --
> eric

Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/






Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message