Mailing-List: contact users-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@activemq.apache.org
Received-SPF: pass (athena.apache.org: domain of rajdavies@gmail.com
 designates 74.125.78.145 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:from:to:in-reply-to:content-type
         :content-transfer-encoding:mime-version:subject:date:references
         :x-mailer;
        b=sn8lEQrliZQ8PxHXzv94/tRN4tBMzH/hMIdfg7wNmqGtnVfzNGZIse3NuuaIXHjBB2
         F7+Ts/+CdC09VyNEwu/aqWqHoSslwj6IHE6KKn/1c8Ct8gCoTmM+MfUVKmZitPusjLQG
         37Jz2ltvdgwWAi8bn8bTLT52VrWFFUjUESj9w=
Message-Id: <44508F78-A457-4A64-A915-82683C829E22@gmail.com>
From: Rob Davies <rajdavies@gmail.com>
To: users@activemq.apache.org
In-Reply-To: <ec6e67fd0901290900k1e56aa54kf97c27f43382a89c@mail.gmail.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v930.3)
Subject: Re: Single consumer for multiple brokers?
Date: Thu, 29 Jan 2009 17:30:05 +0000
References: <15768836.post@talk.nabble.com> <15768993.post@talk.nabble.com>
 <15769001.post@talk.nabble.com> <15769003.post@talk.nabble.com>
 <15769009.post@talk.nabble.com> <21729358.post@talk.nabble.com>
 <3a73c17c0901290813j59f5dd20k2a665df61c42a0ee@mail.gmail.com>
 <aba378a0901290850n7dbf5cc6naa76a97ece8b9154@mail.gmail.com>
 <ec6e67fd0901290900k1e56aa54kf97c27f43382a89c@mail.gmail.com>

http://activemq.apache.org/fanout-transport-reference.html - is the  
place to look to get started
On 29 Jan 2009, at 17:00, James Strachan wrote:

> The fanout transport kinda has most of the implementation code to do
> this; it just needs to be hacked a little so that consume start/stop
> commands are sent to multiple brokers and transactions/acks are
> correctly sent to the broker that sent the original message. There
> might be some fun and games with transactions in there mind; but for
> simple-ish use cases it shouldn't be too hard to hack the
> FanoutTransport if anyone fancies having a go
>
>
> 2009/1/29 Jim Lloyd <jlloyd@silvertailsystems.com>:
>> I'm interested in a variation of this for load balancing. Assume  
>> the volume
>> of data coming from publishers is very large, and you need N  
>> brokers (think
>> N is 10 or more). Furthermore, you want N+1 or N+2 redundancy so  
>> that if any
>> 1 or 2 brokers die, there is sufficient capacity to continue  
>> handling all of
>> the traffic (after affected publishers reconnect). The brokers  
>> might all be
>> behind a hardware load balancer (e.g. F5 or NetScaler) so that all
>> publishers connect via a virtual IP.
>>
>> Now, from the consumer side, we also have many consumers. Each  
>> consumer only
>> subscribes to a subset of the available topics, and we arrange via  
>> design of
>> our topics such that one consumer can always keep up with the  
>> volume of data
>> published on one topic. But the consumer must connect to every  
>> broker.
>>
>> So, we need a fan-in variation of a failover transport that  
>> connects to
>> every broker and actively consumes data from all brokers. If any  
>> broker
>> disconnects, the consumer would use the exponential backoff  
>> reconnect logic
>> to reconnect when the broker becomes available.
>>
>> So, the backup=true option isn't helpful. Instead, we want  
>> something like
>> "fan_in=true".
>>
>> It's not hard to build this kind of fan in logic on top of the  
>> failover
>> transport, but it would be cool if the failover transport was  
>> capable of
>> doing fan in directly.
>>
>> On Thu, Jan 29, 2009 at 8:13 AM, Gary Tully <gary.tully@gmail.com>  
>> wrote:
>>
>>> have you looked at the failover transport? see:
>>> http://activemq.apache.org/failover-transport-reference.html
>>> with the backup=true option, a connection to all listed brokers will
>>> be created so that they are in hot standby in
>>> the event that the first connection is dropped. In this way,  
>>> failover
>>> can be very fast.
>>>
>>> 2009/1/29 kaykay <kaykay.unique@gmail.com>:
>>>>
>>>> This thread is old but just curious if there has been a recent  
>>>> update of
>>> this
>>>> w.r.t ActiveMQ 5.2 . The problem that I am trying to solve is  
>>>> similar
>>> where
>>>> a consumer listens to multiple brokers (as a failover redundancy  
>>>> issue
>>>> instead of listening to a single broker).
>>>>
>>>>
>>>> Stepan Koltsov wrote:
>>>>>
>>>>> I'm playing with 5.0.
>>>>>
>>>>> How do you think, is it hard to write new Transport that  
>>>>> consumes from
>>>>> multiple brokers?
>>>>>
>>>>> S.
>>>>>
>>>>>
>>>>> ttmdev wrote:
>>>>>>
>>>>>> Yes, I got similar results in my consumer testing. What version  
>>>>>> of AMQ
>>>>>> are you using?
>>>>>>
>>>>>> Your only recourse may be to multi thread your consumer and  
>>>>>> have it
>>>>>> create a connection to each of the brokers.
>>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> View this message in context:
>>> http://www.nabble.com/Single-consumer-for-multiple-brokers--tp15768836p21729358.html
>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> http://blog.garytully.com
>>>
>>> Open Source SOA
>>> http://FUSESource.com
>>>
>>
>
>
>
> -- 
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/