Mailing-List: contact users-help@qpid.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@qpid.apache.org
Received-SPF: unknown (nike.apache.org: error in processing during lookup of
 shuston@riverace.com)
From: Steve Huston <shuston@riverace.com>
To: "users@qpid.apache.org" <users@qpid.apache.org>
Subject: RE: Queue mirroring: clustering vs. replication
Thread-Topic: Queue mirroring: clustering vs. replication
Thread-Index: AQHOgu1A29ETNSmzjE28otOltyO8kZlo54mQ
Date: Wed, 17 Jul 2013 14:17:11 +0000
Message-ID: 
 <00BCD2BEF4977149A7C7303B4BAEBBE6504A2D@ORD2MBX02C.mex05.mlsrvr.com>
References: 
 <0ED54485BDF923428E38AB27448A35771A0C8926@de08ex3007.global.ds.honeywell.com>
 <00BCD2BEF4977149A7C7303B4BAEBBE6502B85@ORD2MBX02C.mex05.mlsrvr.com>
 <0ED54485BDF923428E38AB27448A35771A0C8972@de08ex3007.global.ds.honeywell.com>
 <00BCD2BEF4977149A7C7303B4BAEBBE6502CC4@ORD2MBX02C.mex05.mlsrvr.com>
 <0ED54485BDF923428E38AB27448A35771A0C8D46@de08ex3007.global.ds.honeywell.com>
In-Reply-To: 
 <0ED54485BDF923428E38AB27448A35771A0C8D46@de08ex3007.global.ds.honeywell.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Hi Steve,

> -----Original Message-----
> From: Rothkin, Steve (NY81) [mailto:Steve.Rothkin@Honeywell.com]
> Sent: Wednesday, July 17, 2013 8:57 AM
> To: users@qpid.apache.org
> Subject: Queue mirroring: clustering vs. replication
>=20
> > > Subject: RE: Queue mirroring with message grouping and clustering on
> > > Windows
> > >
> > > > > I'm considering a Qpid.22 implementation under MS Windows for
> > > > > message queueing.  In the future we might go to a mixed
> > > > > environment with both Windows and Linux computers.
> > > > > For fault tolerance, I want the queues to be mirrored across 2
> > > > > to
> > > > > 3 computers which are connected by high speed LAN. Each queue
> > > > > will have multiple consumers on different computers (including
> > > > > some NOT hosting the queue), so I also need to use the message
> > > > > grouping feature to ensure that messages from a single source
> > > > > are not processed
> > > out of order.
>=20
> What are the differences between choosing to use clustering to implement
> the solution vs. using ha-replication of individual queues in a non-clust=
ered
> setup?

You summarized it well below, but also, essentially, reusing the capabiliti=
es already present, or reinventing them to suit the particulars of your use=
 case.

> From what I've read so far it seems that with clustering my work would be=
 a
> lot easier as the broker/resource manager combination would take care of
> failover while with non-clustered I'd have to take care of a bunch of thi=
ngs in
> my code -- designating master, detecting failure (which might take longer=
),
> failover (including finding and connecting to the new master).

Right. Don't forget to account for all the ways things can fail. Brokers, n=
odes, network links, etc.

> On the other hand it seems that without clustering, I could pick differen=
t
> masters for different queues (instead of having cluster resource manager
> designate a specific host as master for all of its queues) which would be=
 one
> way of spreading the load of handling client connections.

You could also run multiple clustered brokers on the same nodes and have br=
oker-cluster-1 serve one set of queues and broker-cluster-2 serve a differe=
nt set. I'm not sure about the automatability of keeping the primaries on d=
ifferent nodes, but you could write a script to check it and move the prima=
ry around if needed.

> I could also have
> different operating systems hosting replicas of the same queue (I haven't
> found a cluster resource manager that runs on both Windows and Linux).

That's a great point - clusters are generally uniform.

> I'm also wondering about geographic site mirroring and failover. We have =
2
> sites connected by a WAN. The same services are available at both locatio=
ns.
> Normally a request received at one location would be processed there (and
> sending it to the other site for processing would be more "expensive" as
> would queue mirroring to the other site). But there might be situations
> where one entire site goes down -- if that were to happen it might be goo=
d
> to have the other site detect and finish processing any in-flight message=
s
> from the down site.

There are more factors in this type of design than I'd want to try and cove=
r in email. I'm working with a customer now that is starting to run tests o=
n a multi-site environment, but their goals and constraints and resources m=
ay be quite different from yours. I think you've got a good grasp of the fa=
cilities available (queue dup, federation, clustering) but if you want furt=
her in-depth help, please let me know.

-Steve


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org