activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Strachan" <>
Subject Re: Cluster transport ...
Date Sat, 17 May 2008 09:34:49 GMT
2008/5/15 Sridhar2008 <>:
> Rob/James,
> Thanks for the feedback. I will address both of your questions in a combined
> way  :-)
> Rob> Do you need persistent messaging - or non-persistent only?
> James> or just pick one broker per operation/transaction?
> Initially, it is going to be the above. Since the messaging usage scenarios
> are many, I will just assume the use case of interest for now is durable
> transfer using cluster for high availability.  In this case I am not sure
> that it buys much to send to multiple brokers simultaneously (see my next
> paragraph below) - adds additional headaches as James sites below (however,
> there may be other use cases where this might be useful), in addition to
> making client-side logic complex.


For sending to a topic (outside of a transaction which may include
other operations) then sending the message to all brokers is a no
brainer. I guess so long as acknowledgements only get sent to the
broker they came from & for queue sending we only send to one of the
brokers it should be fairly straight forward.

(So a little bit of hacking to the FanOutTransport should do the trick I think).

We could get clever going forward; where rather than randomly picking
one of the brokers (or round robbin) we kinda partition destinations
across the available brokers?

> Some thoughts: Eventual goal is a broker down in the cluster implies
> capacity hit and not a service hit. So, I will rather solve this use case as
> a distributed storage issue (after 'cluster transport' is added) -
> investigate either a DHT based solution or perhaps something like Hadoop
> with a JDBC interface. The state is replicated in multiple nodes of the
> cluster so the broker that is down can be ignored (ie no need to sweat on
> recovering its state).


FWIW distributing the state is trivial - the FanOutTransport can do
this today really. The issue is ensuring consistency (so more to do
with locking & consistency than moving the messages around).

Incidentally I'm intrigued by the DNS discovery stuff; it sounds great
:). What did you have in mind? That each broker on startup would
register a DNS entry (with a timeout maybe so they get removed) and
clients would ping DNS when attempting to connect in the


Open Source Integration

View raw message