activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Bain <tb...@alumni.duke.edu>
Subject Re: Network of brokers and forwarding performance
Date Wed, 26 Apr 2017 12:56:12 GMT
Auke,

Pooling your connections is definitely a good thing, but since you say that
the system still exhibits the performance slowdown, your next step is what
I said in my first response: to characterize which OS resource (CPU,
network, disk, memory) is the limiting one, and on which broker or client
process.

Thanks for the detailed description of how you're using the network of
brokers; that clears up my confusion.

Tim

On Apr 25, 2017 8:10 AM, "anoppe" <auke@noppe.nl> wrote:

> Hi Tim,Thank you for your replies.I've found one bug in our software: We
> didn't used a PooledConnectionFactory, which caused every message to be
> produced to mq2 setup a new connection to mq2. I've fixed this in the
> meanwhile and repeated my tests; the results were somewhat better (+1m/s),
> but not satisfying enough.To answer your 'BTW' reply: We have split the
> message-flow into two separate parts: mq1 and mq2. We have an API where
> clients sent messages to, which gets published to either mq1 or mq2. The
> responsibility of mq1 is to process 'legacy' messages from the API, and
> mq2's responsibility is to process 'new' messages from the API. The
> datamodel of those messages are exactly the same, but we choose to split
> into separate flows to minimise interference for existing customers.The
> reason we introduced the 'network of brokers'-pattern:We have a number of
> other web-applications/API's that consume messages from 'even.stream.topic'
> (is an aggregate of all topics om mq2 and mq1) from mq1. We didn't want to
> change all these applications to consume from both brokers, therefore we
> decided to let mq2 forward 'company.*.topic'@mq2 contents to mq1. (And
> some
> other topics, but are not relevant for the concept, in my opinion)Recap:-
> All topics that match the name 'company.*.topic'@mq2 are duplicated to
> 'event.stream.topic'@mq1 - The topic 'data.stream.topic'@mq2 is duplicated
> to 'event.stream.topic'@mq1 - The topic 'alert.triggers.forwarder.
> topic'@mq2
> is duplicated to 'alert.triggers.topic'@mq1. - When testing, I publish
> messages to mq2 that will end up in only 1 specific 'company.*.topic'.-
> That
> specific 'company.*.topic'@mq2 gets consumed by an application that
> inserts
> that message into the database, which I measure the consuming-rate on.
> (5m/s)- Another instance of that application is consuming from mq1. (which
> consumes ~30m/s)- If I stop producing measurements on mq2, the rate of
> consuming increases to ~20-25m/s. Which is acceptable.Does this help in
> understanding our situation and the problem we're facing?Regards,Auke Noppe
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/Network-of-brokers-and-forwarding-performance-
> tp4725187p4725223.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message