camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: Painfully slow AMQ consumer
Date Wed, 21 May 2014 05:41:09 GMT
Hi

Try without writing to the file, but just do a

from jms
  to log

and see if that is faster.

As appending to the *same* file name happens sequentially in Camel.



On Wed, May 21, 2014 at 6:06 AM, nw31304 <david.niedzielski@gmail.com> wrote:
> I am attempting to do something really simple: read from an ActiveMQ queue
> and append the messages to a file.  Each message is roughly 100 bytes in
> length and the arrival rate is no more than 1000 messages per minutes.  No
> matter what I try, messages queue up on the broker because the Camel route
> does not process them quickly enough (i.e. append them to the file).  The
> disk I am writing to is SAN attached, an more than capable of these paltry
> requirements. I have ridiculous amounts of CPU at my disposal, an I/O
> utilization and queue time is minimal.  The machine load average on both the
> broker and the machine where I'm running camel is next to nothing, and none
> of the relevant machines are short of memory.  It doesn't matter if I run
> the broker on the same machine or on a different machine, the result is the
> same: a slowly growing queue.  If I tail the file,  literally see messages
> scrolling slowly enough to read them as they're written.  I'm using Camel
> 2.9.0, and ActiveMQ 5.5 (both client and broker).  I thought perhaps it was
> an issue with connection esablishment and teardown, but when I display the
> connections, it appears that the same set of connections are being used.
> Here's the relevant beans and routes:
>
>     <bean id="jmsConnectionFactory"
> class="org.apache.activemq.ActiveMQConnectionFactory">
>           <property name="brokerURL" value="tcp://gsdb1:61616" />
>     </bean>
>
>     <bean id="pooledConnectionFactory"
> class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start"
> destroy-method="stop">
>         <property name="maxConnections" value="20" />
>         <property name="connectionFactory" ref="jmsConnectionFactory" />
>     </bean>
>
>     <bean id="jmsConfig"
> class="org.apache.camel.component.jms.JmsConfiguration">
>         <property name="connectionFactory" ref="pooledConnectionFactory"/>
>         <property name="concurrentConsumers" value="5"/>
>     </bean>
>
>     <bean id="activemq"
> class="org.apache.activemq.camel.component.ActiveMQComponent">
>         <property name="configuration" ref="jmsConfig"/>
>     </bean>
>
>     <camelContext id="camel1" xmlns="http://camel.apache.org/schema/spring">
>
>         <jmxAgent id="agent" createConnector="true" registryPort="20023"/>
>
>         <route>
>             <from uri="activemq:gpsdata.gps.incoming.australia" />
>             <to
> uri="file://../archive-australia?fileName=gpsdata.log&amp;fileExist=Append"
> />
>     </camelContext>
>
>
> Can anyone lend a hand with this, or perhaps give me hand as to where to
> look?
>
> Thanks in advance.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Painfully-slow-AMQ-consumer-tp5751382.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/

Mime
View raw message