qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gibson, Jack" <jagib...@paypal.com.INVALID>
Subject Re: Create a message bus with Qpid
Date Fri, 12 Dec 2014 16:13:24 GMT
Jan -

You can use the dispatch router to build such a topology.  Clients would
then use the router as their endpoint.  The router would have need to be
configured with a balanced configuration with waypoints to each broker.
The result:
     C
     |
     R
    / \
   B   B

The client could use a local broker with again connectors and waypoints to
the router.  If properly setup, you would be able route to/from both local
and remote addresses as well as enable load balancing, fanout, etc
depending on the use case.


Jack





On 12/12/14, 8:06 AM, "jan rinze" <janrinze@gmail.com> wrote:

>Hi all,
>
>It's been quite a struggle to get messages from one queue to a queue on
>another broker.
>The simplest but also most restrictive way is to use
>   qpid-route queue add <dest> <src> <queue>
>however this is simply a point-to-point solution. and setting it up with
>--durable requires much more and often results in error..
>
>Message based service oriented architecture requires services (message
>consumers) to be fed based on specific routing rules or simply by having
>each service type have it's own queue.
>
>A robust message bus will need a local (yes, localhost..) broker that will
>be federated with brokers on other nodes. Both clients and services will
>need local queues that usually keep persistent messages. Thus ensuring
>that
>network issues will not cause failure of either service or client.
>
>a simple setup of three nodes N1,N2 and N3 would require:
>
>client C1,C2 on N1
>service S1,S2 on N2
>service S1,S2 on N3
>
>clients C1 and C2 (on N1) will request service S1 or S2 for a specific
>function by sending a message to local (on N1) queue 'QS1' or 'QS2'.
>Routing will make sure messages from queue 'QS1' (on N1) will be forwarded
>to queue 'QS1' on either N2 or N3 depending on load and availability of
>service S1 on N2 and N3.
>
>How should this work in the broker?
>
>QS1 on N1 holds messages sent by C1 and C2.
>QS1 on N2 and QS1 N3 will receive messages from QS1 on N1 if there is a
>consumer on their queue.
>messages should only be sent to either N2 or N3 to ensure no messages are
>duplicated. (load balancing)
>
>This will allow dynamical upscaling of services on N2 and N3 and a future
>N4 or N5..
>
>Currently there are no options to configure this kind of setup unless
>clustering is used and that is a really 'expensive' setup where all queues
>are replicated everywhere and network disruptions can cause message
>duplication.
>
>There was a proposal to allow topic exchanges to have a load balance
>option
>that would implement a similar scheme. However this is not part of QPID at
>this time.  (see: https://issues.apache.org/jira/browse/QPID-4077 )
>
>If the proposed add-on feature 4077 isn't necessary then how can we
>implement the described scenario with qpid? Also the target environment
>will be many brokers spanning multiple large networks..
>
>Any ideas?
>
>Best regards,
>
>Jan Rinze Peterzon.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org


Mime
View raw message