qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jan rinze <janri...@gmail.com>
Subject Create a message bus with Qpid
Date Fri, 12 Dec 2014 14:06:03 GMT
Hi all,

It's been quite a struggle to get messages from one queue to a queue on
another broker.
The simplest but also most restrictive way is to use
   qpid-route queue add <dest> <src> <queue>
however this is simply a point-to-point solution. and setting it up with
--durable requires much more and often results in error..

Message based service oriented architecture requires services (message
consumers) to be fed based on specific routing rules or simply by having
each service type have it's own queue.

A robust message bus will need a local (yes, localhost..) broker that will
be federated with brokers on other nodes. Both clients and services will
need local queues that usually keep persistent messages. Thus ensuring that
network issues will not cause failure of either service or client.

a simple setup of three nodes N1,N2 and N3 would require:

client C1,C2 on N1
service S1,S2 on N2
service S1,S2 on N3

clients C1 and C2 (on N1) will request service S1 or S2 for a specific
function by sending a message to local (on N1) queue 'QS1' or 'QS2'.
Routing will make sure messages from queue 'QS1' (on N1) will be forwarded
to queue 'QS1' on either N2 or N3 depending on load and availability of
service S1 on N2 and N3.

How should this work in the broker?

QS1 on N1 holds messages sent by C1 and C2.
QS1 on N2 and QS1 N3 will receive messages from QS1 on N1 if there is a
consumer on their queue.
messages should only be sent to either N2 or N3 to ensure no messages are
duplicated. (load balancing)

This will allow dynamical upscaling of services on N2 and N3 and a future
N4 or N5..

Currently there are no options to configure this kind of setup unless
clustering is used and that is a really 'expensive' setup where all queues
are replicated everywhere and network disruptions can cause message
duplication.

There was a proposal to allow topic exchanges to have a load balance option
that would implement a similar scheme. However this is not part of QPID at
this time.  (see: https://issues.apache.org/jira/browse/QPID-4077 )

If the proposed add-on feature 4077 isn't necessary then how can we
implement the described scenario with qpid? Also the target environment
will be many brokers spanning multiple large networks..

Any ideas?

Best regards,

Jan Rinze Peterzon.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message