activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sridhar2008 <activ...@komandur.com>
Subject Re: Cluster transport ...
Date Mon, 19 May 2008 17:13:42 GMT


Thanks for the link. I quickly glanced through the dnsjava website, and not
clear if it requires a specific server for updates to work. 

Also, most large organizations have their own legacy DNS in place, with some
home grown way of updating the DNS records. So we need to accommodate this
use case (for example, making 'auto-registration' optional).


James.Strachan wrote:
> 
> Ah cool thanks for that! I was wondering if an existing DNS library
> might be useful. e.g.
> 
> http://www.dnsjava.org/
> 
> which has a DNS client and server.
> 
> I guess ZeroConf is kinda similar too; just not requiring a new or
> custom DNS server (though at the expense of requiring multicast
> support)
> 
> 2008/5/19 Sridhar2008 <activemq@komandur.com>:
>>> Incidentally I'm intrigued by the DNS discovery stuff; it sounds >great
>>>:). What did you have in mind? That each broker on startup >would
>>>register a DNS entry (with a timeout maybe so they get >removed) and
>>
>> Yes, auto-registration.  Couple of things to note:
>>
>> - The 'timeout feature' again is DNS master specific, the existing ones
>> need
>> to be enhanced to support this aging out. The standard timeout you see in
>> the DNS entry is for a different purpose: refresh between  master and
>> slave
>> DNS servers.
>>
>> - There is no standard APY for web dns  registration, so we will  define
>> an
>> APY, with specific implementatioin to interact with the DNS server (folks
>> can chose to change the implementation for their env).
>>
>> - Most messaging systems  run in trusted env, but some use cases may need
>> authentication before a broker is allowed to do 'auto-registration'. This
>> can be addressed later.
>>
>>>clients would ping DNS when attempting to connect in the
>>>FailoverTransport?
>>
>> Yes, we would specify the cluster name in client config and they would
>> connect using 'ClusterTransport' :-)
>>
>>
>> James.Strachan wrote:
>>>
>>> 2008/5/15 Sridhar2008 <activemq@komandur.com>:
>>>>
>>>>
>>>> Rob/James,
>>>>
>>>> Thanks for the feedback. I will address both of your questions in a
>>>> combined
>>>> way  :-)
>>>>
>>>> Rob> Do you need persistent messaging - or non-persistent only?
>>>> James> or just pick one broker per operation/transaction?
>>>>
>>>> Initially, it is going to be the above. Since the messaging usage
>>>> scenarios
>>>> are many, I will just assume the use case of interest for now is
>>>> durable
>>>> transfer using cluster for high availability.  In this case I am not
>>>> sure
>>>> that it buys much to send to multiple brokers simultaneously (see my
>>>> next
>>>> paragraph below) - adds additional headaches as James sites below
>>>> (however,
>>>> there may be other use cases where this might be useful), in addition
>>>> to
>>>> making client-side logic complex.
>>>
>>> Agreed.
>>>
>>> For sending to a topic (outside of a transaction which may include
>>> other operations) then sending the message to all brokers is a no
>>> brainer. I guess so long as acknowledgements only get sent to the
>>> broker they came from & for queue sending we only send to one of the
>>> brokers it should be fairly straight forward.
>>>
>>> (So a little bit of hacking to the FanOutTransport should do the trick I
>>> think).
>>>
>>> We could get clever going forward; where rather than randomly picking
>>> one of the brokers (or round robbin) we kinda partition destinations
>>> across the available brokers?
>>>
>>>
>>>
>>>> Some thoughts: Eventual goal is a broker down in the cluster implies
>>>> capacity hit and not a service hit. So, I will rather solve this use
>>>> case
>>>> as
>>>> a distributed storage issue (after 'cluster transport' is added) -
>>>> investigate either a DHT based solution or perhaps something like
>>>> Hadoop
>>>> with a JDBC interface. The state is replicated in multiple nodes of the
>>>> cluster so the broker that is down can be ignored (ie no need to sweat
>>>> on
>>>> recovering its state).
>>>
>>> Agreed.
>>>
>>> FWIW distributing the state is trivial - the FanOutTransport can do
>>> this today really. The issue is ensuring consistency (so more to do
>>> with locking & consistency than moving the messages around).
>>>
>>> Incidentally I'm intrigued by the DNS discovery stuff; it sounds great
>>> :). What did you have in mind? That each broker on startup would
>>> register a DNS entry (with a timeout maybe so they get removed) and
>>> clients would ping DNS when attempting to connect in the
>>> FailoverTransport?
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://open.iona.com
>>>
>>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Cluster-transport-...-tp17221068s2354p17323314.html
>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>>
>>
> 
> 
> 
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> Open Source Integration
> http://open.iona.com
> 
> 

-- 
View this message in context: http://www.nabble.com/Cluster-transport-...-tp17221068s2354p17323601.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.


Mime
View raw message