Mailing-List: contact users-help@camel.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@camel.apache.org
Received-SPF: pass (nike.apache.org: domain of claus.ibsen@gmail.com
 designates 209.85.220.212 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type:content-transfer-encoding;
        b=u41sMcLWe4Qra5jl4KjW8wFlBV559bvp4i0bn9D1gGyl2efIojjLezLfPLIA8hqCs5
         Jk6uMeeOtV5dn1+8jtepn4HOvtgehNI35giVumMU2bE2ASVdie1Ks/9zpUcZukX75sQo
         b0vrk2wufNh0XgivRbjRrJYdZJA286l51hLeI=
MIME-Version: 1.0
In-Reply-To: <24198880.post@talk.nabble.com>
References: <24198880.post@talk.nabble.com>
From: Claus Ibsen <claus.ibsen@gmail.com>
Date: Thu, 25 Jun 2009 10:40:18 +0200
Message-ID: <5380c69c0906250140u42db42bfq9cef446126a0c1c4@mail.gmail.com>
Subject: Re: Faster!!
To: users@camel.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi

Thanks a lot for sharing this great benchmark with us.

Camel was not designed with ultra fast performance in mind. It was
after all making integration easier.
So there is of course a trade off when Camel does the routing for you.

However we are paving the road in post Camel 2.0 timeframe to have the
underlying architecture refactored allowing us to improve the
performance.

Hadrian is currently working on this. See ticket:
https://issues.apache.org/activemq/browse/CAMEL-1078

Yes the type converter had an issue in Camel 1.5.0 that impacted performanc=
e.
We have address this since.
There could be a few minor performance gains having some eager checks
before resorting down the type converter chain.

We have also thought of allowing Camel to bootstrap all its
components/converters etc. on startup so it can do pre check and have
it loaded.
Loading all the type converters do take some time.

A few minor tips that could help performance.

1)
Disable error handling. Configure Camel to use the NoErrorHandler.
We should probably have a better switch to disable it instead of
having a noop error handler woven in the route.
That will be optimized when we get the Channel logic being more
intelligent and dynamic.

2)
Disable JMX as it instruments and measure each and every message being rout=
ed

3)
Does Camel log alot while you test? I assume INFO level logging does
not produce noise in the log?

4)
And then the Spring JMS listener itself can be optimized.
http://static.springframework.org/spring/docs/2.5.x/api/org/springframework=
/jms/listener/DefaultMessageListenerContainer.html#setCacheLevel(int)


We love contributions and help. If anyone having time to sit down with
a profiler and check bottlenecks and contention points with Camel that
would be awesome.

In camel-core there is a very simply unit test that outputs a simple
performance on a route: RoutePerformanceTest
Can be used as a start to profile and check spots.


On Thu, Jun 25, 2009 at 10:08 AM, bwtaylor <bryan_w_taylor@yahoo.com> wrote=
:
>
> I've been doing the simplest possible performance/benchmark test on Camel=
. I
> took a stock activemq 5.2 with camel 1.5 and the example.A to example.B
> route, bumped up the queue memory to 50m and ran
> =A0ant producer -Dmax=3D10000 -Dverbose=3Dfalse -Dsubject=3Dexample.A
> from the example directory. It took about 38 seconds for the camel route =
to
> finish handing them to example.B.
>
> Next I updated to camel 2.0-M2, I ran the same A to B route in about 38
> seconds again.
> 2.0-M2, with no persistence store finished in 28 seconds.
> 2.0-M2, with no persistence store and a route of activemq:example.A to
> browse:example.B took 24 seconds.
> A control load with no persistence took 6 seconds: ant producer -Dmax=3D1=
0000
> -Dverbose=3Dfalse -Dsubject=3Dexample.C
>
> Assuming the browse endpoint write is negligible, this leaves 24-6=3D18
> seconds for activemq to hand messages to camel and camel to ask it's next
> endpoint to take them. Presumably the activemq dequeue is no worse than a
> write, so camel is introducing 12 seconds to move 10k messages.
> Unfortunately, for one of operations I'm working on right now, a complex
> event processing problem where I'm using esper, Camel seems to be an
> insurmountable speed bottleneck for me. Esper is rocking fast and in my
> testing I was pumping 20M pojos in and getting 4M out in 25 seconds. It
> looks impossible to get more than about 800 messages per second through
> Camel, regardless of what you connect to. This is really annoying, becaus=
e
> camel makes writing esper glue code extremely nice!
>
> Based on the numbers above, camel actually seems comparable to adding
> activemq persistence. That doesn't seem reasonable: Camel's speed for sim=
ple
> from-to routes seems to be the wrong order of magnitude. When I've profil=
ed
> in the past, I've seen TypeConverter stuff showing up as the bottleneck. =
I
> don't know if that's still the case or not, but it's a good example where
> speed should be able to be substituted for cleverness. I'd love to see Ca=
mel
> add a set of benchmarks, and publish the numbers every release. On the
> prominent benchmarks, I'd love to see every release of Camel having a tic=
ket
> or two addressing the top bottlenecks. Surely I'm not the only user out
> there interested in seeing a Camel get Faster.
> --
> View this message in context: http://www.nabble.com/Faster%21%21-tp241988=
80p24198880.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>


--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus