activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Bain <tb...@alumni.duke.edu>
Subject Re: Network of brokers and forwarding performance
Date Tue, 25 Apr 2017 13:14:20 GMT
Quoting from an email I posted in another thread a few days ago:

When a system is slow, there's always something (or several somethings,
rotating between them) that is the limiting resource. I'd start by trying
to determine what that resource is, using standard OS monitoring tools (top
and the like). If you can tell that CPU is a primary bottleneck, use a
profiler or sampler like JVisualVM to determine where the time is being
spent, and share what you find here (maybe there's a bug we need to fix).
If it's disk IO, make sure your Log4J logging is at the right level (i.e.
not DEBUG or TRACE), and if it's still the bottleneck, consider options to
upgrade your storage hardware to something more performant. If the JVM is
spending a lot of time garbage collecting (JVisualVM and JConsole will tell
you that), increase the broker's heap size. If your OS is heavily into
swap, buy more RAM. And the list goes on, but the starting point is you
characterizing what's actually slow, and we can help you figure out why
it's slow.

Tim

On Apr 24, 2017 2:15 AM, "anoppe" <auke@noppe.nl> wrote:

> Hi everybody,
>
> I've setup 2 brokers with different responsibilities:
> mq1 is an instance the webapplications consumes messages from and is
> receiving messages also
> mq2 is an instance that received messages only.
>
> mq2 has some composite topics configured:
>         <destinationInterceptors>
>           <virtualDestinationInterceptor>
>             <virtualDestinations>
>               <compositeTopic name="company.*.topic" forwardOnly="false">
>                 <forwardTo>
>                   <topic physicalName="event.stream.topic" />
>                 </forwardTo>
>               </compositeTopic>
>               <compositeTopic name="data.stream.topic" forwardOnly="false">
>                 <forwardTo>
>                   <topic physicalName="event.stream.topic" />
>                 </forwardTo>
>               </compositeTopic>
>               <compositeTopic name="alert.triggers.forwarder.topic"
> forwardOnly="true">
>                 <forwardTo>
>                   <topic physicalName="alert.triggers.topic" />
>                 </forwardTo>
>               </compositeTopic>
>             </virtualDestinations>
>           </virtualDestinationInterceptor>
>         </destinationInterceptors>
>
> - All topics that match the name 'company.*.topic' are duplicated to
> 'event.stream.topic'
> - The topic 'data.stream.topic' is duplicated to 'event.stream.topic'
> - The topic 'alert.triggers.forwarder.topic' is duplicated to
> 'alert.triggers.topic'.
>
> mq1 and mq2 are in a network of brokers, where mq2 is forwarding the
> messages in 'event.stream.topic' and 'alert.triggers.topic' to mq1, so the
> consumers on that instance receive those messages also. To prevent
> duplication, I've excluded 'data.stream.topic' form the network-of-brokers:
> <networkConnectors>
>     <networkConnector name="eventStreamAndAlertTriggersForwarder"
> uri="static:(tcp://ip-of-mq1:61616)" duplex="false">
>         <excludedDestinations>
>                 <topic physicalName="data.stream.topic" />
>         </excludedDestinations>
>         <staticallyIncludedDestinations>
>              <topic physicalName="event.stream.topic"/>
>              <topic physicalName="alert.triggers.topic"/>
>         </staticallyIncludedDestinations>
>     </networkConnector>
> </networkConnectors>
>
> On a functional level this works just fine. But, we tried to loadtest our
> infrastructure and ran into unexpected behaviour: When we send messages at
> a
> rate of ~200m/s to 'company.*.topic'@mq2, consuming from mq2 becomes very
> slow: When we stop sending new messages, we are able to consume at a rate
> of
> ~30m/s. If we start sending messages again, the rate of consuming drops to
> ~5m/s and a queue builds up.
> The consumer we measured runs on the same machine.
> When we send and consume messages on mq1, we get the expected rate of about
> 30m/s.
>
> Does anyone has an idea how we can improve performance? Did I misconfigure
> something?
>
> Regards,
> Auke Noppe
>
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/Network-of-brokers-and-forwarding-performance-tp4725187.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message