Mailing-List: contact dev-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@activemq.apache.org
Received-SPF: pass (nike.apache.org: domain of lists@nabble.com designates
 216.139.236.158 as permitted sender)
Message-ID: <17323314.post@talk.nabble.com>
Date: Mon, 19 May 2008 10:00:07 -0700 (PDT)
From: Sridhar2008 <activemq@komandur.com>
To: dev@activemq.apache.org
Subject: Re: Cluster transport ...
In-Reply-To: <ec6e67fd0805170234v1f4f4404r9eeb1b073073e480@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
References: <17221068.post@talk.nabble.com>
 <46B44A74-DEB6-4667-8BEA-A5799ACE632D@gmail.com>
 <ec6e67fd0805140145x12b596ecx5296a80bc4a69669@mail.gmail.com>
 <17246093.post@talk.nabble.com>
 <ec6e67fd0805170234v1f4f4404r9eeb1b073073e480@mail.gmail.com>


> Incidentally I'm intrigued by the DNS discovery stuff; it sounds >great
>:). What did you have in mind? That each broker on startup >would
>register a DNS entry (with a timeout maybe so they get >removed) and

Yes, auto-registration.  Couple of things to note:

- The 'timeout feature' again is DNS master specific, the existing ones need
to be enhanced to support this aging out. The standard timeout you see in
the DNS entry is for a different purpose: refresh between  master and slave
DNS servers.

- There is no standard APY for web dns  registration, so we will  define an
APY, with specific implementatioin to interact with the DNS server (folks
can chose to change the implementation for their env).

- Most messaging systems  run in trusted env, but some use cases may need 
authentication before a broker is allowed to do 'auto-registration'. This
can be addressed later.

>clients would ping DNS when attempting to connect in the
>FailoverTransport?

Yes, we would specify the cluster name in client config and they would
connect using 'ClusterTransport' :-)


James.Strachan wrote:
> 
> 2008/5/15 Sridhar2008 <activemq@komandur.com>:
>>
>>
>> Rob/James,
>>
>> Thanks for the feedback. I will address both of your questions in a
>> combined
>> way  :-)
>>
>> Rob> Do you need persistent messaging - or non-persistent only?
>> James> or just pick one broker per operation/transaction?
>>
>> Initially, it is going to be the above. Since the messaging usage
>> scenarios
>> are many, I will just assume the use case of interest for now is durable
>> transfer using cluster for high availability.  In this case I am not sure
>> that it buys much to send to multiple brokers simultaneously (see my next
>> paragraph below) - adds additional headaches as James sites below
>> (however,
>> there may be other use cases where this might be useful), in addition to
>> making client-side logic complex.
> 
> Agreed.
> 
> For sending to a topic (outside of a transaction which may include
> other operations) then sending the message to all brokers is a no
> brainer. I guess so long as acknowledgements only get sent to the
> broker they came from & for queue sending we only send to one of the
> brokers it should be fairly straight forward.
> 
> (So a little bit of hacking to the FanOutTransport should do the trick I
> think).
> 
> We could get clever going forward; where rather than randomly picking
> one of the brokers (or round robbin) we kinda partition destinations
> across the available brokers?
> 
> 
> 
>> Some thoughts: Eventual goal is a broker down in the cluster implies
>> capacity hit and not a service hit. So, I will rather solve this use case
>> as
>> a distributed storage issue (after 'cluster transport' is added) -
>> investigate either a DHT based solution or perhaps something like Hadoop
>> with a JDBC interface. The state is replicated in multiple nodes of the
>> cluster so the broker that is down can be ignored (ie no need to sweat on
>> recovering its state).
> 
> Agreed.
> 
> FWIW distributing the state is trivial - the FanOutTransport can do
> this today really. The issue is ensuring consistency (so more to do
> with locking & consistency than moving the messages around).
> 
> Incidentally I'm intrigued by the DNS discovery stuff; it sounds great
> :). What did you have in mind? That each broker on startup would
> register a DNS entry (with a timeout maybe so they get removed) and
> clients would ping DNS when attempting to connect in the
> FailoverTransport?
> 
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> Open Source Integration
> http://open.iona.com
> 
> 

-- 
View this message in context: http://www.nabble.com/Cluster-transport-...-tp17221068s2354p17323314.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.