Mailing-List: contact users-help@camel.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@camel.apache.org
Received-SPF: pass (nike.apache.org: domain of iocanel@gmail.com designates
 209.85.160.173 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=iWRh3ddGoIld9LHIY8uJyRe1veAhtmdN9LvZXnrD0VKT91PWtPs5blBFpvXyP1ge7E
         sPIRTp/L5w5X6Kdx7yF1z2CtIFQjof12AgCimKJq4izYlAXmfdIMX9GbVkW9j7KtQeRf
         2xJ9imAqlO6LiytVANkhPJHcW8BnKbsG33VX0=
MIME-Version: 1.0
In-Reply-To: <AANLkTi=wmJAanVYU=zRLnVQ=fLjkFz1PbC6aF8LZJRNf@mail.gmail.com>
References: <AANLkTik46dNhe4rJG3Ad67J5wVJVx3ar3++BsmqnTuyH@mail.gmail.com>
	<AANLkTi==0iWC0636V63VHX4rEhCfbTtAkipOxkXQVYib@mail.gmail.com>
	<AANLkTi=qsFXy3Mz2=JmiLHjAUj_fpfK1UZkpfVXe3udp@mail.gmail.com>
	<AANLkTi=zdjxJm_4YxkrKO1geH479Ytp9Ezqt=WC3Kaev@mail.gmail.com>
	<AANLkTi=yRikbxrj08VhRzqv1Ht=X-HiAP5RFP4A6+Fyh@mail.gmail.com>
	<AANLkTimyCLp_Uvw-FMuabkT-uPo0Vgs3V3Z5LwWtM0Vw@mail.gmail.com>
	<AANLkTi=wmJAanVYU=zRLnVQ=fLjkFz1PbC6aF8LZJRNf@mail.gmail.com>
Date: Thu, 9 Dec 2010 15:07:05 +0200
Message-ID: <AANLkTimr5q8O8VOmD04RphJ54FATY6s+bQ_5q+a+rq11@mail.gmail.com>
Subject: Re: Performance issues when using camel-activemq and transactions.
From: Ioannis Canellos <iocanel@gmail.com>
To: users@camel.apache.org
Content-Type: multipart/alternative; boundary=20cf3054961114faf10496f9ea5f

--20cf3054961114faf10496f9ea5f
Content-Type: text/plain; charset=ISO-8859-1

Hi Claus,

Here are the results of the tests I run:

Producer Only:  10msg/sec (transacted=true) / 2800 msg/sec
(transacted=false).
Consumer Only: 10msg/sec transacted=true) / ~500 msg/sec (transacted=false).

The results of the mixed is the ones I initially posted.

The difference is so big and I have tried almost everything, but whenever I
set "transacted=true" there is nothing on earth that can give me the
slightest performance increase.


On Thu, Dec 9, 2010 at 11:47 AM, Claus Ibsen <claus.ibsen@gmail.com> wrote:

> When using transacted=true then the commit is sync, and thus the
> consumer side is impacted.
>
> So you should be able to send as fast as you can. But the consumer
> side is slower due the begin/commit pair in the transaction is
> synchronous.
>
> If you use camel-jms on the consumer side to consume and process the
> messages, then its based on top of Spring JMS.
> And all the logic and code is synchronous and thus slower.
>
> Maybe you can do 3 performance tests.
> a - send only to the queue
> b - consume only from the queue (the queue is preloaded)
> c - both send and consume at the same time
>
>
>
> On Thu, Dec 9, 2010 at 10:28 AM, Ioannis Canellos <iocanel@gmail.com>
> wrote:
> > I think that when I set "transacted=true" then it ignores the property
> > "useAsyncSend=true" set on the connection factory.
> >
> > I am not sure if those two properties can be used together, but they
> > probably can't and this why I have performance issues.
> >
> >
> > On Tue, Dec 7, 2010 at 3:46 PM, Ioannis Canellos <iocanel@gmail.com>
> wrote:
> >
> >> I will do so.
> >>
> >> I will also check the tx scenarios described in the "Camel in Action".
> >>
> >>
> >>
> >> On Tue, Dec 7, 2010 at 3:36 PM, Claus Ibsen <claus.ibsen@gmail.com>
> wrote:
> >>
> >>> Check some of the TX unit tets in camel-jms and compare notes.
> >>>
> >>>
> >>> On Tue, Dec 7, 2010 at 2:20 PM, Ioannis Canellos <iocanel@gmail.com>
> >>> wrote:
> >>> > Hi Claus,
> >>> >
> >>> > thanks for your response.
> >>> >
> >>> > I had the luck to watch that webinar and I also have the slides, the
> do
> >>> help
> >>> > in increasing the overall performance. But when I set
> transaction=true
> >>> on
> >>> > the activemq component, the performance can degrade form 2000 msg/sec
> to
> >>> 5
> >>> > msg/sec.
> >>> >
> >>> > Regarding the number of connections, even if I increase it to 10, 20
> or
> >>> more
> >>> > I don't see any difference at all.
> >>> >
> >>> > I feel that the problem is in my camel configuration (not activemq)
> >>> since I
> >>> > didn't have issues when I was use servicemix and activemq.
> >>> >
> >>> > Thanks again!
> >>> >
> >>> > On Tue, Dec 7, 2010 at 3:08 PM, Claus Ibsen <claus.ibsen@gmail.com>
> >>> wrote:
> >>> >
> >>> >> Ask at AMQ forum as its generally how to optimize and setup AMQ.
> >>> >>
> >>> >> Also check out maybe some of the webinars by Rob Davies on advanced
> >>> >> and high performance AMQ stuff
> >>> >> The webinars is avail at fusesource website.
> >>> >>
> >>> >> And it looks like you only got 1 connection in your pool?
> >>> >> maxConnections=1
> >>> >>
> >>> >>
> >>> >>
> >>> >> On Mon, Nov 29, 2010 at 10:38 AM, Ioannis Canellos <
> iocanel@gmail.com>
> >>> >> wrote:
> >>> >> > I am using camel 2.5.0 and activemq 5.4.0 to create a simple route
> of
> >>> the
> >>> >> > form dataset -> activemq:queue.in -> activemq:queue.out ->
> dataset.
> >>> >> > All are working fine and I the performance is more than 1K
> >>> messages/sec.
> >>> >> >
> >>> >> > However if I set transactions=true on the activemq component
> >>> >> configuration,
> >>> >> > then I experience serious performance issues (5 messages/sec).
> >>> >> >
> >>> >> > I am testing using camel:run maven plugin. My configuration is the
> >>> >> > following:
> >>> >> >
> >>> >> > <beans
> >>> >> >        xmlns="http://www.springframework.org/schema/beans"
> >>> >> >        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> >>> >> >        xmlns:amq="http://activemq.apache.org/schema/core"
> >>> >> >        xmlns:camel="http://camel.apache.org/schema/spring"
> >>> >> >        xmlns:context="
> http://www.springframework.org/schema/context"
> >>> >> >        xmlns:aop="http://www.springframework.org/schema/aop"
> >>> >> >        xsi:schemaLocation="
> >>> http://www.springframework.org/schema/beans
> >>> >> > http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
> >>> >> >        http://activemq.apache.org/schema/core
> >>> >> > http://activemq.apache.org/schema/core/activemq-core.xsd
> >>> >> >        http://www.springframework.org/schema/context
> >>> >> >
> http://www.springframework.org/schema/context/spring-context-3.0.xsd
> >>> >> >        http://www.springframework.org/schema/aop
> >>> >> > http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
> >>> >> >        http://camel.apache.org/schema/spring
> >>> >> > http://camel.apache.org/schema/spring/camel-spring.xsd">
> >>> >> >
> >>> >> >    <!-- Allows us to use system properties as variables in this
> >>> >> > configuration file -->
> >>> >> >    <bean
> >>> >> >
> >>> >>
> >>>
> class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"/>
> >>> >> >
> >>> >> >    <!-- ActiveMQ Broker -->
> >>> >> >    <broker id="broker" xmlns="
> http://activemq.apache.org/schema/core
> >>> "
> >>> >> > brokerName="localhost" dataDirectory="data"
> >>> >> >            useJmx="true" persistent="true"
> advisorySupport="false">
> >>> >> >        <destinationPolicy>
> >>> >> >            <policyMap>
> >>> >> >                <policyEntries>
> >>> >> >                    <policyEntry queue=">"
> producerFlowControl="false"
> >>> >> > optimizedDispatch="true">
> >>> >> >                        <pendingQueuePolicy>
> >>> >> >                            <vmQueueCursor/>
> >>> >> >                        </pendingQueuePolicy>
> >>> >> >                    </policyEntry>
> >>> >> >                </policyEntries>
> >>> >> >            </policyMap>
> >>> >> >        </destinationPolicy>
> >>> >> >
> >>> >> >        <persistenceAdapter>
> >>> >> >            <kahaDB directory="data/kahadb"
> >>> enableIndexWriteAsync="true"/>
> >>> >> >        </persistenceAdapter>
> >>> >> >
> >>> >> >        <transportConnectors>
> >>> >> >            <transportConnector name="nio" uri="nio://
> 0.0.0.0:61616
> >>> "/>
> >>> >> >        </transportConnectors>
> >>> >> >    </broker>
> >>> >> >
> >>> >> >    <!-- Camel Context -->
> >>> >> >    <camelContext xmlns="http://camel.apache.org/schema/spring"
> >>> >> > id="messaging" depends-on="broker">
> >>> >> >        <route>
> >>> >> >            <from uri="dataset:loadtest?produceDelay=-1"/>
> >>> >> >            <to uri="activemq:queue:queue.in"/>
> >>> >> >        </route>
> >>> >> >    </camelContext>
> >>> >> >
> >>> >> >    <!-- Camel Components -->
> >>> >> >    <bean id="activemq"
> >>> >> > class="org.apache.activemq.camel.component.ActiveMQComponent">
> >>> >> >        <property name="connectionFactory"
> >>> ref="pooledConnectionFactory"/>
> >>> >> >        <!--property name="transacted" value="true"/>
> >>> >> >        <property name="transactionManager"
> ref="transactionManager"/>
> >>> >> >        <property name="transactionTimeout" value="300000"/-->
> >>> >> >        <property name="acknowledgementModeName"
> >>> >> value="AUTO_ACKNOWLEDGE"/>
> >>> >> >        <property name="cacheLevelName" value="CACHE_AUTO"/>
> >>> >> >    </bean>
> >>> >> >
> >>> >> >
> >>> >> >    <bean id="connectionFactory"
> >>> >> > class="org.apache.activemq.ActiveMQConnectionFactory">
> >>> >> >        <property name="brokerURL" value="nio://127.0.0.1:61616"/>
> >>> >> >        <property name="useAsyncSend" value="true"/>
> >>> >> >    </bean>
> >>> >> >
> >>> >> >    <bean id="pooledConnectionFactory"
> >>> >> > class="org.apache.activemq.pool.PooledConnectionFactory">
> >>> >> >        <property name="connectionFactory"
> ref="connectionFactory"/>
> >>> >> >        <property name="maxConnections" value="1"/>
> >>> >> >        <property name="maximumActive" value="500"/>
> >>> >> >    </bean>
> >>> >> >
> >>> >> >    <!-- DataSet -->
> >>> >> >    <bean id="loadtest"
> >>> >> > class="org.apache.camel.component.dataset.SimpleDataSet">
> >>> >> >        <property name="size" value="100000"/>
> >>> >> >    </bean>
> >>> >> >
> >>> >> >    <!-- Transaction Manager -->
> >>> >> >    <bean id="transactionManager"
> >>> >> > class="org.jencks.factory.TransactionManagerFactoryBean"/>
> >>> >> > </beans>
> >>> >> >
> >>> >> >
> >>> >> > --
> >>> >> > *Ioannis Canellos*
> >>> >> > http://iocanel.blogspot.com
> >>> >> > Integration Engineer @ Upstream S.A. <
> http://www.upstreamsystems.com
> >>> >
> >>> >> >
> >>> >>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> Claus Ibsen
> >>> >> -----------------
> >>> >> FuseSource
> >>> >> Email: cibsen@fusesource.com
> >>> >> Web: http://fusesource.com
> >>> >> Twitter: davsclaus
> >>> >> Blog: http://davsclaus.blogspot.com/
> >>> >> Author of Camel in Action: http://www.manning.com/ibsen/
> >>> >>
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > *Ioannis Canellos*
> >>> > http://iocanel.blogspot.com
> >>> >
> >>> > Integration Engineer @ Upstream S.A. <http://www.upstreamsystems.com
> >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> Claus Ibsen
> >>> -----------------
> >>> FuseSource
> >>> Email: cibsen@fusesource.com
> >>> Web: http://fusesource.com
> >>> Twitter: davsclaus
> >>> Blog: http://davsclaus.blogspot.com/
> >>> Author of Camel in Action: http://www.manning.com/ibsen/
> >>>
> >>
> >>
> >>
> >> --
> >> *Ioannis Canellos*
> >> http://iocanel.blogspot.com
> >>
> >> Integration Engineer @ Upstream S.A. <http://www.upstreamsystems.com>
> >>
> >>
> >>
> >
> >
> > --
> > *Ioannis Canellos*
> > http://iocanel.blogspot.com
> >
> > Integration Engineer @ Upstream S.A. <http://www.upstreamsystems.com>
> >
>
>
>
> --
> Claus Ibsen
> -----------------
> FuseSource
> Email: cibsen@fusesource.com
> Web: http://fusesource.com
> Twitter: davsclaus
> Blog: http://davsclaus.blogspot.com/
> Author of Camel in Action: http://www.manning.com/ibsen/
>


-- 
*Ioannis Canellos*
http://iocanel.blogspot.com

Integration Engineer @ Upstream S.A. <http://www.upstreamsystems.com>

--20cf3054961114faf10496f9ea5f--