camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xenofon Papadopoulos <xenofon.papadopou...@upstreamsystems.com>
Subject Re: ActiveMQ routes performance
Date Sat, 04 Aug 2012 08:48:36 GMT
Thanks for the tips. I'm trying, however, to understand why there is better
behavior with the intermediate queue, given that all other parameters
(threads etc) remain the same.

Also this is a test; in production we normally use about 800 concurrent
consumers, and the resource use in the servers is pretty low.


On Sat, Aug 4, 2012 at 6:44 AM, Ben O'Day <ben.oday@initekconsulting.com>wrote:

> tough to say given the unknowns about your setup, but here are a couple of
> things to consider...
>
> -for kahadb, set enableJournalDiskSyncs to false to get much better
> throughput
> -try fewer consumer threads...100 is a lot and you generally have
> diminished/negative results with larger thread counts
> -tune your CachingConnectionFactory (notably the sessionCacheSize should be
> increased from the default)...that said, beware of using this with polling
> consumers connections (see
>
> http://stackoverflow.com/questions/5916638/autoreconnect-problem-with-activemq-and-cachingconnectionfactory
> )
> -alternatively, try using the AMQ PooledConnectionFactory and see if you
> get better performance
> -see this thread on AMQ performance:
>
> http://stackoverflow.com/questions/7463415/activemq-performance-gotchas-and-precautions
>
>
> -Ben
>
>
>
> On Fri, Aug 3, 2012 at 8:44 AM, Xenofon Papadopoulos <
> xenofon.papadopoulos@upstreamsystems.com> wrote:
>
> > Sorry in advance if I should have posted that in ActiveMQ list, but here
> is
> > our case.
> > We are running the same test with two different setups:
> >
> > Setup 1:
> > ------------
> > We are using a single ActiveMQ broker and a single Camel with the routes:
> >
> > from( "jetty://http://..." ).inOnly( "activemq:queue:queue2" )
> >
> > from("activemq:queue:queue2?maxConcurrentConsumers=100").bean(
> myProcessor,
> > "delay" )
> >
> > The activeMQ component uses
> > a org.springframework.jms.connection.CachingConnectionFactory
> >
> > myProcess.delay() delays for 200 ms
> >
> >
> > Setup 2:
> > ------------
> > We are using two ActiveMQ brokers and a single Camel with the routes:
> >
> > from( "jetty://http://..." ).inOnly( "activemq1:queue:queue1" )
> >
> > from(
> > "activemq1:queue:queue1?maxConcurrentConsumers=100").inOnly(
> > "activemq2:queue:queue2"
> > )
> >
> > from("activemq2:queue:queue2?maxConcurrentConsumers=100").bean(
> > myProcessor, "delay" )
> >
> > Both activeMQ components use their
> > own org.springframework.jms.connection.CachingConnectionFactory
> > ------------------------
> >
> > All ActiveMQ brokers are identical, with:
> >
> > - kahadb
> > - <policyEntry queue=">" queuePrefetch="20" producerFlowControl="false"
> > optimizedDispatch="true" useCache="true">
> > - <memoryUsage limit="4 gb"/>
> > - <transportConnector name="openwire" uri="tcp://0.0.0.0:xxxx"/>
> >
> > We test the setup by sending http requests from jmeter with 40 threads
> and
> > 1200 tps filter. We monitor the enqueue and dequeue rates of all the
> > queues, and notice that:
> >
> > Setup 1:
> > ------------
> > queue2 enqueue: ~900 tps
> > queue2 dequeue: ~50 tps
> >
> >
> > Setup 2:
> > ------------
> > queue1 enqueue: ~1200 tps
> > queue1 dequeue: ~1200 tps
> > queue2 enqueue: ~1200 tps
> > queue2 dequeue: ~150 tps
> >
> > This means that the 2nd setup causes the brokers to behave much better
> and
> > gives us a much higher throughput for our processor.
> > How can this be explained? Is there any configuration that will let us
> > achieve the same performance with a single broker?
> >
> > Thank you
> > Xenofon
> >
>



-- 
*Xenofon Papadopoulos
*
*Upstream
*4 Kastorias & Messinias Street
153 44 Gerakas Attikis
Mob +30 694 027 5392
DL   +30 210 661 8277
Fax +30 210 661 8550
www.upstreamsystems.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message