activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marc Zampetti <>
Subject Re: Questions on Network of Brokers and high message rates
Date Wed, 05 Dec 2007 16:03:01 GMT


Yes, it sounds like the JEDI thing and the partitioning approach is what I
need. And yes, I'm talking queues for the most part.  For the partitioning,
is that something that AMQ, or are you talking about me having a layer in
front of AMQ that would do this. In this case, instead of having one big
network of brokers, I would have several smaller networks of brokers?


James.Strachan wrote:
> On 05/12/2007, Marc Zampetti <> wrote:
>> James,
>> You are right about the HA comment I originally made. I was referring to
>> fact that I'm not looking for persistent messages. But I am concerned
>> about
>> what happens when a broker fails and being able to recover from that
>> quickly, even if that means losing messages.
> Master/Slave is all about replicating messages to avoid message loss -
> it comes at a heavy performance cost. If you can't afford that cost,
> you don't need master/slave. i.e. the normal failover of the JMS
> client will do fine.
>> I understand the point about the need for more information. Here is what
>> I
>> say so far.
>> There will be around 30 - 50 processes producing messages, each one
>> producing the same type of messages. Think it of a farm of processes
>> doing
>> the same basic work, spread out to handle processing load, scalability,
>> fault tolerance, etc. These processes know nothing about what happens to
>> these messages once they send them.
>> On the consuming side, there are multiple farms of processes, each farm
>> doing a particular type of processing on some subset of the messages.
>> There
>> is nothing in the message, other than a unique id, that can be used to
>> determine which messages should be sent to each farm. Each of these farms
>> would have anywhere from 2 - 30 processes, depending upon the
>> functionality
>> and load requirements.
> So we're talking queues right? And by the sounds of it you could
> partition messages across multiple brokers for scalability. (Sounds
> like a good use case for the JEDI transport that still needs
> completing BTW to stripe message flows across different brokers to
> avoid the downsides of the store and forward network of brokers).
>> For the AMQ topologies, I was thinking of a Virtual Topic that all of the
>> publishers send messages to, with queues behind that go to some sort of
>> routing layer that in turns sends the message to the queue that the
>> service
>> the relevant consumer farm.
> Yeah. I guess it depends on how complex the routing is. It might be
> easier to just have an input queue, then some consumer process (which
> could be hosted inside the broker if need be) to do the routing;
> whether its Camel or some custom JMS code or whatever.
> -- 
> James
> -------
> Open Source Integration

View this message in context:
Sent from the ActiveMQ - User mailing list archive at

View raw message