activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Pavlovich <mattr...@gmail.com>
Subject Re: Persistent message througput when using sessions/connections
Date Wed, 23 Oct 2013 22:19:33 GMT
Hi Michal-

Lot of stuff to try to cover here.. but in short.. performance tuning is a tough thing to
cover in an email.

1. Separate the testing of perf through the broker from your app, so you can get a baseline.
 Look into the activemq-perf tooling.

2. 40 producers and 40 consumers into one queue is probably too many.. look to cut that way
down to like 2-4 per CPU core.

3. Spring integration is difficult to tune b/c it masks the connection, session handling.
Be sure to read up on CACHE_CONSUMER, CACHE_SESSION and CACHE_CONNECTION

4. Are you using transactions? 

5. If you have a real-time messaging pattern, look into the vmCursor as an option

6. Test every change one at a time after you get your baseline.

7. Tuning for one destination is very different than tuning for 100's or 1,000's of destinations.

8. Turn on nio, confirm you aren't hitting memory limits.. turn producer flow control back
on (its helpful to know when you are hitting an upper limit)

9. Don't produce and consume over the same session.. separate producers and consumers (and
connections)

Hope this helps.

-Matt

On Oct 23, 2013, at 5:36 AM, gmicky <michal.gubik@gmail.com> wrote:

> Hello,
> 
> we are trying to achieve hight througput using persistent messages together
> with sync send and queues. However we realised there are some limitations
> that could be maybe resolved or explained but you.
> 
> Scenario 1 - Speed ok , unusable with springs integration
> Consumers and producers are connected using multiple separate connections
> /40 producers , 40  consumers per queue/ . With this scenario we achieve
> very high throughput of cca 500 000 messages per minute. However are
> framework is springs and it only allows concurent connections on one
> connection and several sessions bases.
> 
> Scenario 2 - Producing with multiple connections no consumers 
> Producers connected multiple separate connections same as in scenario 1 but
> no consumers connected. I would expect rate of writing to kahadb to be same
> as in scenario 1 but it does not happen. Message production rate is at cca
> 18 000msg/minute. I do not understand this at all as messages in scenarion 1
> also have to pass persistent store but do so at much much higher rates when
> they have consumers connected with dedicated connections.
> 
> Scenario 3 -
> One connection and multiple sessions for consumers and same for producers /1
> connection - 40 consumer and 1 connection and 40 producer sessions. This
> works fine with springs integration but the performace is again very slow at
> cca 18 000msg/minute.
> 
> Producer flow contor is switched off and no memory limits on queue are
> imposed so throttling is not an option. Leaving springs integration and
> using multiple connections for producers and consumers is also not a option.
> Having such low througput as in scenario 2) and scenario 3) is also not a
> option. 
> 
> Could you please advice what is the problem with using 1 connection and
> multiple session for high throughput is it amq thread model? I can imagine
> some limits due to single socket used but I would not think the impact on
> throughput would be so huge. Also could you please explain why producing
> with no consumers is so slow compared to producing with multiple consumers
> connected each by own connection?
> 
> Thanks for answers
> 
> 
> 
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/Persistent-message-througput-when-using-sessions-connections-tp4673145.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message