qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Huston <shus...@riverace.com>
Subject RE: Queue mirroring: clustering vs. replication
Date Wed, 17 Jul 2013 14:17:11 GMT
Hi Steve,

> -----Original Message-----
> From: Rothkin, Steve (NY81) [mailto:Steve.Rothkin@Honeywell.com]
> Sent: Wednesday, July 17, 2013 8:57 AM
> To: users@qpid.apache.org
> Subject: Queue mirroring: clustering vs. replication
> 
> > > Subject: RE: Queue mirroring with message grouping and clustering on
> > > Windows
> > >
> > > > > I'm considering a Qpid.22 implementation under MS Windows for
> > > > > message queueing.  In the future we might go to a mixed
> > > > > environment with both Windows and Linux computers.
> > > > > For fault tolerance, I want the queues to be mirrored across 2
> > > > > to
> > > > > 3 computers which are connected by high speed LAN. Each queue
> > > > > will have multiple consumers on different computers (including
> > > > > some NOT hosting the queue), so I also need to use the message
> > > > > grouping feature to ensure that messages from a single source
> > > > > are not processed
> > > out of order.
> 
> What are the differences between choosing to use clustering to implement
> the solution vs. using ha-replication of individual queues in a non-clustered
> setup?

You summarized it well below, but also, essentially, reusing the capabilities already present,
or reinventing them to suit the particulars of your use case.

> From what I've read so far it seems that with clustering my work would be a
> lot easier as the broker/resource manager combination would take care of
> failover while with non-clustered I'd have to take care of a bunch of things in
> my code -- designating master, detecting failure (which might take longer),
> failover (including finding and connecting to the new master).

Right. Don't forget to account for all the ways things can fail. Brokers, nodes, network links,
etc.

> On the other hand it seems that without clustering, I could pick different
> masters for different queues (instead of having cluster resource manager
> designate a specific host as master for all of its queues) which would be one
> way of spreading the load of handling client connections.

You could also run multiple clustered brokers on the same nodes and have broker-cluster-1
serve one set of queues and broker-cluster-2 serve a different set. I'm not sure about the
automatability of keeping the primaries on different nodes, but you could write a script to
check it and move the primary around if needed.

> I could also have
> different operating systems hosting replicas of the same queue (I haven't
> found a cluster resource manager that runs on both Windows and Linux).

That's a great point - clusters are generally uniform.

> I'm also wondering about geographic site mirroring and failover. We have 2
> sites connected by a WAN. The same services are available at both locations.
> Normally a request received at one location would be processed there (and
> sending it to the other site for processing would be more "expensive" as
> would queue mirroring to the other site). But there might be situations
> where one entire site goes down -- if that were to happen it might be good
> to have the other site detect and finish processing any in-flight messages
> from the down site.

There are more factors in this type of design than I'd want to try and cover in email. I'm
working with a customer now that is starting to run tests on a multi-site environment, but
their goals and constraints and resources may be quite different from yours. I think you've
got a good grasp of the facilities available (queue dup, federation, clustering) but if you
want further in-depth help, please let me know.

-Steve


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org


Mime
View raw message