qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Ross <tr...@redhat.com>
Subject Re: [AMQP 1.0] Basic question on loadsharing due to high volume
Date Mon, 11 Jan 2016 18:10:10 GMT
Hi Morgan,

Your issue is one of the major drivers behind the Qpid Dispatch Router 
project (http://qpid.apache.org/components/dispatch-router/index.html). 
  The architectural premise behind Dispatch Router is that message 
routing and message brokering/queuing are separate activities that 
should be handled by separate components.

One of the key use-cases for the router is to place one or more routers 
between the messaging clients and the brokers.  This provides several 
scaling benefits:

  - Messages that do not require queuing may be transferred directly
    from endpoint to endpoint without passing through a broker.  This
    transfer may be multicast (all subscribers for an address receive
    one copy of every produced message) or anycast (every produced
    message is delivered to one subscriber).
  - Multiple brokers can provide queues and/or topics on the same
    address.  The router network can then distribute message traffic
    across the brokers allowing for horizontal scaling without a single
    choke point for message flow.

I'd be happy to provide more detail if you are interested.



On 01/08/2016 10:34 AM, Morgan Lindqvist wrote:
> Hi All,
> I have a little problem understanding how load sharing is supported with
> AMQP. Up till now that has not been an issue for me but now I will start
> hitting the capacity on the HW where the server is running.
> I have a number of producer and a number of consumers. The AMQP server in
> the middle now have such a high load that it does not cope processing more
> requests. What I would like to do is to solve this on the AMQP server side
> without affecting the clients. I do however not understand how this can be
> done.
> The description on
> http://qpid.apache.org/releases/qpid-cpp-0.34/cpp-broker/book/chap-Messaging_User_Guide-Broker_Federation.html
> talk about using Federation for load balancing. My understanding is that all
> the producers still talk to one AMQP instance that then forward the requests
> to "slave" servers. When you hit the capacity of this one single instance
> how does one scale the system even more?
> When the load is shared between multiple slave AMQP servers, are then all
> the consumers required to connect to each of these slave servers to fetch
> the messages? So if the load is increased and I need to add another AMQP
> slave server all the consumers need to be re-configured to also fetch
> messages from this one?
> Is there an other way that is better for scaling a system when one have a
> lot of data?
> Best Regards,
> Morgan
> --
> View this message in context: http://qpid.2158936.n2.nabble.com/AMQP-1-0-Basic-question-on-loadsharing-due-to-high-volume-tp7636285.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org

To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

View raw message