activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From anoppe <a...@noppe.nl>
Subject Re: Network of brokers and forwarding performance
Date Tue, 25 Apr 2017 13:59:13 GMT
Hi Tim,Thank you for your replies.I've found one bug in our software: We
didn't used a PooledConnectionFactory, which caused every message to be
produced to mq2 setup a new connection to mq2. I've fixed this in the
meanwhile and repeated my tests; the results were somewhat better (+1m/s),
but not satisfying enough.To answer your 'BTW' reply: We have split the
message-flow into two separate parts: mq1 and mq2. We have an API where
clients sent messages to, which gets published to either mq1 or mq2. The
responsibility of mq1 is to process 'legacy' messages from the API, and
mq2's responsibility is to process 'new' messages from the API. The
datamodel of those messages are exactly the same, but we choose to split
into separate flows to minimise interference for existing customers.The
reason we introduced the 'network of brokers'-pattern:We have a number of
other web-applications/API's that consume messages from 'even.stream.topic'
(is an aggregate of all topics om mq2 and mq1) from mq1. We didn't want to
change all these applications to consume from both brokers, therefore we
decided to let mq2 forward 'company.*.topic'@mq2 contents to mq1. (And some
other topics, but are not relevant for the concept, in my opinion)Recap:-
All topics that match the name 'company.*.topic'@mq2 are duplicated to
'event.stream.topic'@mq1 - The topic 'data.stream.topic'@mq2 is duplicated
to 'event.stream.topic'@mq1 - The topic 'alert.triggers.forwarder.topic'@mq2
is duplicated to 'alert.triggers.topic'@mq1. - When testing, I publish
messages to mq2 that will end up in only 1 specific 'company.*.topic'.- That
specific 'company.*.topic'@mq2 gets consumed by an application that inserts
that message into the database, which I measure the consuming-rate on.
(5m/s)- Another instance of that application is consuming from mq1. (which
consumes ~30m/s)- If I stop producing measurements on mq2, the rate of
consuming increases to ~20-25m/s. Which is acceptable.Does this help in
understanding our situation and the problem we're facing?Regards,Auke Noppe



--
View this message in context: http://activemq.2283324.n4.nabble.com/Network-of-brokers-and-forwarding-performance-tp4725187p4725223.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message