qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gordon Sim <g...@redhat.com>
Subject Re: New cluster design (maybe of interest to MessageStore implementaors too)
Date Fri, 24 Sep 2010 16:11:24 GMT
On 09/22/2010 10:35 PM, Alan Conway wrote:
> I'm proposing a redesign of the C++ cluster, this is how I would have
> done it in the first place if only I had known...
> Here's an initial writeup:
>
> https://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/qpid/cluster/new-cluster-design.txt

I like your ideas; sounds like the right direction to me. Revising the 
interface/mechanism around asynchronous completion of enqueue/dequeue as 
used by the store seems like one key first step.

There are two components to this design, (a) replication of message 
state and (b) coordination of the queue locking/ownership. Keeping these 
separate and behind minimal interfaces will I think allow for a lot of 
flexibility.

For example there is nothing in the message state replication that 
requires multicast or even virtual synchrony, so it could be possible to 
use that same interface for replication over a WAN to a DR site 
(eliminating the need for the so called 'asynchronous queue state 
replication' we have partially implemented now).

Also the queue ownership mechanism in the simplest case can allow for 
active-passive configuration; with ownership transferring only on 
failure of the primary.

The similarity between replication and persistence of message state is 
also worth pursuing. Can the store and the cluster share the same 
interface to the queue? Tracking acquisition as a distinct step is not 
something the store does at present, but at some level (at some point?) 
at least it could be useful.

On more specific comment:

> *** Requirements
>
> The cluster must provide these delivery guarantees:
>
> - client sends transfer: message must be replicated and not lost even if the local broker
crashes.
> - client acquires a message: message must not be delivered on another broker while acquired.
> - client rejects acquired message: message must be re-queued on cluster and not lost.

That should perhaps be 'releases' rather than 'rejects'? Reject is also 
an interesting case, but currently rejected messages are not requeued, 
they are delivered to the alter-nate exchange for the queue if defined 
(i.e. the dead-lettering mechanism defined in 0-10) or dropped if not.

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org


Mime
View raw message