activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Strachan" <james.strac...@gmail.com>
Subject Re: Cluster transport ...
Date Mon, 19 May 2008 17:03:33 GMT
Ah cool thanks for that! I was wondering if an existing DNS library
might be useful. e.g.

http://www.dnsjava.org/

which has a DNS client and server.

I guess ZeroConf is kinda similar too; just not requiring a new or
custom DNS server (though at the expense of requiring multicast
support)

2008/5/19 Sridhar2008 <activemq@komandur.com>:
>> Incidentally I'm intrigued by the DNS discovery stuff; it sounds >great
>>:). What did you have in mind? That each broker on startup >would
>>register a DNS entry (with a timeout maybe so they get >removed) and
>
> Yes, auto-registration.  Couple of things to note:
>
> - The 'timeout feature' again is DNS master specific, the existing ones need
> to be enhanced to support this aging out. The standard timeout you see in
> the DNS entry is for a different purpose: refresh between  master and slave
> DNS servers.
>
> - There is no standard APY for web dns  registration, so we will  define an
> APY, with specific implementatioin to interact with the DNS server (folks
> can chose to change the implementation for their env).
>
> - Most messaging systems  run in trusted env, but some use cases may need
> authentication before a broker is allowed to do 'auto-registration'. This
> can be addressed later.
>
>>clients would ping DNS when attempting to connect in the
>>FailoverTransport?
>
> Yes, we would specify the cluster name in client config and they would
> connect using 'ClusterTransport' :-)
>
>
> James.Strachan wrote:
>>
>> 2008/5/15 Sridhar2008 <activemq@komandur.com>:
>>>
>>>
>>> Rob/James,
>>>
>>> Thanks for the feedback. I will address both of your questions in a
>>> combined
>>> way  :-)
>>>
>>> Rob> Do you need persistent messaging - or non-persistent only?
>>> James> or just pick one broker per operation/transaction?
>>>
>>> Initially, it is going to be the above. Since the messaging usage
>>> scenarios
>>> are many, I will just assume the use case of interest for now is durable
>>> transfer using cluster for high availability.  In this case I am not sure
>>> that it buys much to send to multiple brokers simultaneously (see my next
>>> paragraph below) - adds additional headaches as James sites below
>>> (however,
>>> there may be other use cases where this might be useful), in addition to
>>> making client-side logic complex.
>>
>> Agreed.
>>
>> For sending to a topic (outside of a transaction which may include
>> other operations) then sending the message to all brokers is a no
>> brainer. I guess so long as acknowledgements only get sent to the
>> broker they came from & for queue sending we only send to one of the
>> brokers it should be fairly straight forward.
>>
>> (So a little bit of hacking to the FanOutTransport should do the trick I
>> think).
>>
>> We could get clever going forward; where rather than randomly picking
>> one of the brokers (or round robbin) we kinda partition destinations
>> across the available brokers?
>>
>>
>>
>>> Some thoughts: Eventual goal is a broker down in the cluster implies
>>> capacity hit and not a service hit. So, I will rather solve this use case
>>> as
>>> a distributed storage issue (after 'cluster transport' is added) -
>>> investigate either a DHT based solution or perhaps something like Hadoop
>>> with a JDBC interface. The state is replicated in multiple nodes of the
>>> cluster so the broker that is down can be ignored (ie no need to sweat on
>>> recovering its state).
>>
>> Agreed.
>>
>> FWIW distributing the state is trivial - the FanOutTransport can do
>> this today really. The issue is ensuring consistency (so more to do
>> with locking & consistency than moving the messages around).
>>
>> Incidentally I'm intrigued by the DNS discovery stuff; it sounds great
>> :). What did you have in mind? That each broker on startup would
>> register a DNS entry (with a timeout maybe so they get removed) and
>> clients would ping DNS when attempting to connect in the
>> FailoverTransport?
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://open.iona.com
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Cluster-transport-...-tp17221068s2354p17323314.html
> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>
>



-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://open.iona.com

Mime
View raw message