qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Huston <shus...@riverace.com>
Subject RE: Queue mirroring with message grouping and clustering on Windows
Date Tue, 16 Jul 2013 16:03:55 GMT
HA queue replication doesn't help you with load balancing. Clients are not allowed to add or
remove messages from replicas.

> -----Original Message-----
> From: Rothkin, Steve (NY81) [mailto:Steve.Rothkin@Honeywell.com]
> Sent: Tuesday, July 16, 2013 11:45 AM
> To: users@qpid.apache.org
> Subject: RE: Queue mirroring with message grouping and clustering on
> Windows
> 
> For what I'm trying to do, is using HA Queue replication between standalone
> (NOT clustered) nodes a viable option (one that will give me the high
> availability/reliability, load balancing, message grouping, and failover)? Or is
> the clustering required for the message grouping feature to work?
> 
> At http://qpid.apache.org/releases/qpid-0.22/cpp-broker/book/ha-queue-
> replication.html it states "When the replica queue is on the active node of a
> cluster, there is no automatic failover. Automatic failover will be supported in
> the future, in the meantime you can use the workaround detailed in Section
> 1.11.12, "Failover of federation links and queue replication."."
> But it doesn't say anything about failover for queues on unclustered nodes --
> is that automatic or is the same (or some other) workaround required?
> 
> > > > From: Steve Huston [mailto:shuston@riverace.com]
> > > > > I'm considering a Qpid.22 implementation under MS Windows for
> > > > > message queueing.  In the future we might go to a mixed
> > > > > environment with both Windows and Linux computers.
> > > > > For fault tolerance, I want the queues to be mirrored across 2
> > > > > to
> > > > > 3 computers which are connected by high speed LAN. Each queue
> > > > > will have multiple consumers on different computers (including
> > > > > some NOT hosting the queue), so I also need to use the message
> > > > > grouping feature to ensure that messages from a single source
> > > > > are not processed
> > > out of order.
> > > >
> > > > Ok.
> > > >
> > > > > 1)   Is clustering required to do this (on RabbitMQ it is, but RabbitMQ
> > > appears
> > > > > to not support message grouping yet)?
> > > >
> > > > For mirroring across a set of nodes for FT, yes.
> > > >
> > > > > 2)   What is required to use clustering on Windows (and is it even
> > > available)?
> > > > > So far I've read that Corosync is required for clustering and in
> > > > > another place (a few years old) I read that Corosync isn't
> > > > > ported to/doesn't build on Windows.
> > > >
> > > > The new HA module in Qpid requires integration with a resource
> > > > manager; on Linux this is rgmanager (corosync is involved to
> > > > manage the cluster itself, but it's not directly involved with Qpid).
> > > > Currently there is no integration with a resource manager on
> > > > Windows clusters. It's probably not a gigantic amount of work to
> > > > get it there, but it's
> > > work that is needed.
> > >
> > > If there is a mix of linux and windows computers hosting the queues,
> > > do they need to use the same (or a compatible) resource manager?
> >
> > Yes, the brokers participating in the cluster do need to use the same
> resource manager.
> >
> > > As a way of getting around the need for a resource manager on
> > > Windows, if I made a cluster of linux machines (or linux VMs running
> > > under
> > > Windows) to host the mirrored queues would it then be ok/supported
> > > to have windows clients (NOT part of the cluster) to write to and read
> from the queues?
> >
> > Absolutely.
> >
> > > If I wanted to try to do the windows resource manager
> > > implemtentation (assuming my boss would allow it, and given that I
> > > have zero experience developing within Qpid), is there existing
> > > documentation that makes it clear (from the Qpid point of view) what
> > > I need to add to the code. If so, pointers to where to look would be
> helpful.
> >
> > You wouldn't necessarily need to add anything to Qpid - my
> > understanding is that it's more of a Windows scripting/integration
> > issue and not so much a Qpid coding issue. You can check this book:
> > http://qpid.apache.org/releases/qpid-0.22/cpp-broker/book/chapter-ha.h
> > tml Section 1.11.10 has the info to get you going.
> >
> > > BTW, while the queues have to be HA/fault tolerant, I'm expecting
> > > that most of them DON'T need the messages to be persistent because
> > > the messages also have a timeliness aspect to them. So if all the
> > > hosting nodes go down at the same time, it may not make sense to be
> > > able to recover the messages if it takes more than a few minutes for
> > > one of them to come
> > back up.
> >
> > That's going to make your life much easier.
> >
> > -Steve
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org For additional
> commands, e-mail: users-help@qpid.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org


Mime
View raw message