Return-Path: 
 <users-return-48797-apmail-camel-users-archive=camel.apache.org@camel.apache.org>
X-Original-To: apmail-camel-users-archive@www.apache.org
Delivered-To: apmail-camel-users-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id DDA0A11291
	for <apmail-camel-users-archive@www.apache.org>;
 Wed, 21 May 2014 04:07:20 +0000 (UTC)
Received: (qmail 67569 invoked by uid 500); 21 May 2014 04:07:20 -0000
Delivered-To: apmail-camel-users-archive@camel.apache.org
Received: (qmail 67506 invoked by uid 500); 21 May 2014 04:07:20 -0000
Mailing-List: contact users-help@camel.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:users-help@camel.apache.org>
List-Unsubscribe: <mailto:users-unsubscribe@camel.apache.org>
List-Post: <mailto:users@camel.apache.org>
List-Id: <users.camel.apache.org>
Reply-To: users@camel.apache.org
Delivered-To: mailing list users@camel.apache.org
Received: (qmail 67498 invoked by uid 99); 21 May 2014 04:07:20 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 May 2014 04:07:20 +0000
X-ASF-Spam-Status: No, hits=2.3 required=5.0
	tests=SPF_SOFTFAIL,URI_HEX
X-Spam-Check-By: apache.org
Received-SPF: softfail (athena.apache.org: transitioning domain of
 david.niedzielski@gmail.com does not designate 216.139.236.26 as permitted
 sender)
Received: from [216.139.236.26] (HELO sam.nabble.com) (216.139.236.26)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 May 2014 04:07:16 +0000
Received: from [192.168.236.26] (helo=sam.nabble.com)
	by sam.nabble.com with esmtp (Exim 4.72)
	(envelope-from <david.niedzielski@gmail.com>)
	id 1Wmxnr-0001zg-MA
	for users@camel.apache.org; Tue, 20 May 2014 21:06:55 -0700
Date: Tue, 20 May 2014 21:06:55 -0700 (PDT)
From: nw31304 <david.niedzielski@gmail.com>
To: users@camel.apache.org
Message-ID: <1400645215681-5751382.post@n5.nabble.com>
Subject: Painfully slow AMQ consumer
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-Virus-Checked: Checked by ClamAV on apache.org

I am attempting to do something really simple: read from an ActiveMQ queue
and append the messages to a file.  Each message is roughly 100 bytes in
length and the arrival rate is no more than 1000 messages per minutes.  No
matter what I try, messages queue up on the broker because the Camel route
does not process them quickly enough (i.e. append them to the file).  The
disk I am writing to is SAN attached, an more than capable of these paltry
requirements. I have ridiculous amounts of CPU at my disposal, an I/O
utilization and queue time is minimal.  The machine load average on both the
broker and the machine where I'm running camel is next to nothing, and none
of the relevant machines are short of memory.  It doesn't matter if I run
the broker on the same machine or on a different machine, the result is the
same: a slowly growing queue.  If I tail the file,  literally see messages
scrolling slowly enough to read them as they're written.  I'm using Camel
2.9.0, and ActiveMQ 5.5 (both client and broker).  I thought perhaps it was
an issue with connection esablishment and teardown, but when I display the
connections, it appears that the same set of connections are being used. 
Here's the relevant beans and routes:

    <bean id="jmsConnectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory">
          <property name="brokerURL" value="tcp://gsdb1:61616" />
    </bean>

    <bean id="pooledConnectionFactory"
class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start"
destroy-method="stop">
        <property name="maxConnections" value="20" />
        <property name="connectionFactory" ref="jmsConnectionFactory" />
    </bean>

    <bean id="jmsConfig"
class="org.apache.camel.component.jms.JmsConfiguration">
        <property name="connectionFactory" ref="pooledConnectionFactory"/>
        <property name="concurrentConsumers" value="5"/>
    </bean>

    <bean id="activemq"
class="org.apache.activemq.camel.component.ActiveMQComponent">
        <property name="configuration" ref="jmsConfig"/>
    </bean>

    <camelContext id="camel1" xmlns="http://camel.apache.org/schema/spring">

        <jmxAgent id="agent" createConnector="true" registryPort="20023"/>

        <route>
            <from uri="activemq:gpsdata.gps.incoming.australia" />
            <to
uri="file://../archive-australia?fileName=gpsdata.log&amp;fileExist=Append"
/>
    </camelContext>


Can anyone lend a hand with this, or perhaps give me hand as to where to
look?  

Thanks in advance.



--
View this message in context: http://camel.465427.n5.nabble.com/Painfully-slow-AMQ-consumer-tp5751382.html
Sent from the Camel - Users mailing list archive at Nabble.com.