camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: Performance and MessageSupport.getBody (1.6.0)
Date Wed, 04 Mar 2009 08:24:18 GMT
Hi

I am looking into this now. Its an issue introduced by the StreamCache
eg:
No type converter available to convert from type: java.lang.Integer to
the required type: org.apache.camel.StreamCache with value 1

Will create a ticket and get onto it

On Tue, Mar 3, 2009 at 6:04 PM, Claus Ibsen <claus.ibsen@gmail.com> wrote:
> Hi
>
> Did we see which type it tries to convert to? You have only said it
> throws 10 exceptions but it should log which type it wanted to convert
> it to.
> That will also help.
>
>
> On Tue, Mar 3, 2009 at 4:35 PM, Claus Ibsen <claus.ibsen@gmail.com> wrote:
>> On Tue, Mar 3, 2009 at 4:19 PM, paquettd <dan.paquette@lmco.com> wrote:
>>>
>>> Not only avoiding the try..catch but just running through all the reflection
>>> code over and over again can't be cheap either.
>> It has a local cache so the lookup will be fast as its a map.
>>
>> But its the throws .. catch exception that is expensive if it does it
>> 10 times for your test case.
>> We gotta check into this. Maybe there is something obvious wrong.
>>
>> We can later improve it to not thrown the excpetion in certain
>> situations where we dont care.
>>
>>
>>>
>>>
>>> willem.jiang wrote:
>>>>
>>>> Maybe we could add some kind of type converter cache to avoid the  try
>>>> ... catch work to improve the performance.
>>>>
>>>> Willem
>>>>
>>>> paquettd wrote:
>>>>> Thank you for trying to help. I'm attaching my config file here. As you
>>>>> can
>>>>> see this is a contrived example I made to test performance (I used a
>>>>> similar
>>>>> example with some other products).
>>>>>
>>>>> Essentially what's happening is a String is being sent with the
>>>>> ProducerTemplate send method; setting the type to InOut. The string is
>>>>> checked for lower case characters; and if it has any it goes to the
>>>>> "capitalizer"; after that it goes to a reverser and truncator before
>>>>> finally
>>>>> a timestamp is put into the header (before I sent it I put a timestamp
as
>>>>> well). I run this a few thousand times; using System.nanoTime for those
>>>>> timestamps (while not real accurate I used the same thing with other
COTS
>>>>> and the relative differences are in time are larger than the clock
>>>>> granularity)
>>>>>
>>>>> After I started seeing long processing times I used JProbe from Quest
>>>>> Software to narrow down to the bottleneck we are talking about. JProbe
>>>>> tells
>>>>> me that Exception is thrown 10 times processing one of these messages....
>>>>> so
>>>>> when I send 1000 messages 10,000 of those exceptions are being thrown
and
>>>>> it
>>>>> takes a very long time for the 1000 messages to get through.
>>>>>
>>>>>
>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>> <beans xmlns="http://www.springframework.org/schema/beans"
>>>>>      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>>
>>>>> xmlns:camel="http://activemq.apache.org/camel/schema/spring"
>>>>>      xsi:schemaLocation="http://www.springframework.org/schema/beans
>>>>>
>>>>> http://www.springframework.org/schema/beans/spring-beans.xsd
>>>>>      http://activemq.apache.org/camel/schema/spring
>>>>> http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
>>>>>
>>>>>      <bean id="reverser" class="camel.test.StringReverser" />
>>>>>      <bean id="capitalizer" class="camel.test.StringCapitalizer"
/>
>>>>>      <bean id="truncater" class="camel.test.StringTruncater">
>>>>>              <property name="length" value="10" />
>>>>>      </bean>
>>>>>         <bean id="endTimestamper"
>>>>> class="camel.test.transformers.HeaderTimestamper">
>>>>>             <property name="headerKey" value="END_TIME"/>
>>>>>         </bean>
>>>>>
>>>>>      <camelContext id="camel.test.camelcontext"
>>>>>              xmlns="http://activemq.apache.org/camel/schema/spring">
>>>>>              <template id="myCamelTemplate" defaultEndpoint="direct:test"
/>
>>>>>
>>>>>              <route>
>>>>>                      <from uri="direct:test" />
>>>>>                      <choice>
>>>>>                              <when>
>>>>>                                      <simple>${body}
regex '.*[a-x].*'</simple>
>>>>>                                      <to uri="direct:capitalizeFirst"
/>
>>>>>                              </when>
>>>>>                              <otherwise>
>>>>>                                      <to uri="direct:reverseAndTruncate"
/>
>>>>>                              </otherwise>
>>>>>                      </choice>
>>>>>              </route>
>>>>>
>>>>>              <route>
>>>>>                      <from uri="direct:capitalizeFirst"
/>
>>>>>                      <bean ref="capitalizer" />
>>>>>                      <to uri="direct:reverseAndTruncate"/>
>>>>>              </route>
>>>>>
>>>>>              <route>
>>>>>                      <from uri="direct:reverseAndTruncate"
/>
>>>>>                      <bean ref="reverser" />
>>>>>                      <bean ref="truncater" />
>>>>>                      <to uri="bean:endTimestamper?method=timestamp"/>
>>>>>              </route>
>>>>>      </camelContext>
>>>>>
>>>>>      <bean id="testDriver" class="camel.test.TestDriver">
>>>>>              <property name="producerTemplate" ref="myCamelTemplate"
/>
>>>>>      </bean>
>>>>> </beans>
>>>>>
>>>>>
>>>>>
>>>>> Claus Ibsen-2 wrote:
>>>>>> On Tue, Mar 3, 2009 at 1:17 AM, Willem Jiang <willem.jiang@gmail.com>
>>>>>> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> If you don't want the TypeConverter to get invovled , you could
use
>>>>>>> MessageSupport.getBody() directly.
>>>>>> Yes I am wondering if he has a .convertBodyTo in the route, so we
need
>>>>>> to see this.
>>>>>>
>>>>>> Or some other endpoint/producer trying to get the body as a special
>>>>>> type, eg InputStream etc.
>>>>>>
>>>>>> But the route would really help.
>>>>>>
>>>>>>
>>>>>>> Willem
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Mar 3, 2009 at 1:43 AM, paquettd <dan.paquette@lmco.com>
wrote:
>>>>>>>
>>>>>>>> I'm not sure if it makes a difference but I'm not using JMS
anywhere.
>>>>>>>> In
>>>>>>>> fact
>>>>>>>> in this test everything is using "direct".
>>>>>>>>
>>>>>>>> Is there something I can do in the Spring DSL to hint to
Camel that
>>>>>>>> there
>>>>>>>> is
>>>>>>>> no conversion necessary?
>>>>>>>>
>>>>>>>>
>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>> On Mon, Mar 2, 2009 at 5:54 PM, paquettd <dan.paquette@lmco.com>
>>>>>>>> wrote:
>>>>>>>>>> I've been seeing some performance problems with Camel
1.6.0 (I have
>>>>>>>> not
>>>>>>>>>> tried
>>>>>>>>>> this with previous versions yet).
>>>>>>>>>>
>>>>>>>>>> My profiler is pointing the finger at MessageSupport.getBody,
>>>>>>>>>> TypeConverter.convertTo, and DefaultTypeConverter.findTypeConverter
>>>>>>>>>> specifically
>>>>>>>>>>
>>>>>>>>>> findTypeConverter is always throwing a
>>>>>>>>>> NoTypeConversionAvailableException;
>>>>>>>>>> which is then being caught and ignored in MessageSupport.getBody;
at
>>>>>>>>>> which
>>>>>>>>>> point processing continues successfully.
>>>>>>>>> This should be normal in situations where you ask the
body to be a
>>>>>>>>> specific type which cannot be converted to.
>>>>>>>>> To help this can you show your route and what kind of
JMS messages
>>>>>>>>> are
>>>>>>>>> you sending.
>>>>>>>>>
>>>>>>>>> Camel is payload type agnostics (eg dont have to be pure
XML etc.) so
>>>>>>>>> you can send whatever objects you like.
>>>>>>>>> It has a rich type converter registry to be able to convert
seamless
>>>>>>>>> between types.
>>>>>>>>>
>>>>>>>>> This registry is loaded on demand, so you should make
sure your start
>>>>>>>>> profiling after Camel is "warm".
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> protected <T> T getBody(Class<T> type,
Object body) is the specific
>>>>>>>>>> getBody
>>>>>>>>>> in question.
>>>>>>>>>>
>>>>>>>>>> Is this exception an expected behavior? It's weird
how the catch
>>>>>>>> block
>>>>>>>>>> doesn't even log a warning. Should a converter have
been found? My
>>>>>>>>>> message
>>>>>>>>>> payload is just a java.lang.String.
>>>>>>>>> In the old days it returned null, but that did not work
as the
>>>>>>>>> payload
>>>>>>>>> you were trying to convert could be null, so it was a
catch-22
>>>>>>>>> situation.
>>>>>>>>> So we added the exception.
>>>>>>>>>
>>>>>>>>> But if throwing exception is expensive we could maybe
add a has test
>>>>>>>>> to avoid this exception being thrown and caught in the
>>>>>>>>> MessageSupport.
>>>>>>>>>
>>>>>>>>> The exception is also meant for end users so they get
a good
>>>>>>>>> exception
>>>>>>>>> detailing the problem if they try to convert something
into eg,
>>>>>>>>> MyFooClass and its not possible to convert to it. Instead
of
>>>>>>>>> returning
>>>>>>>>> a null value as result.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> I suspect I've done something wrong but I don't know
where to start
>>>>>>>>>> looking.
>>>>>>>>>> I'm concerned with this; as I'm comparing Camel to
some other
>>>>>>>>>> message
>>>>>>>>>> routing solutions. This is making Camel take 40 times
longer than
>>>>>>>>>> the
>>>>>>>>>> competition and I want to make sure I do a fair comparison.
>>>>>>>>> We are currently rewamping the internal API in Camel
2.0 that leads
>>>>>>>>> us
>>>>>>>>> up to a point where we can do performance improvements
when Camel
>>>>>>>>> routes exchanges.
>>>>>>>>> Currently it does a bit of defensive copying when it
moves message
>>>>>>>>> from node to node. The revamped API lets us do some more
clever stuff
>>>>>>>>> there to improve the speed.
>>>>>>>>>
>>>>>>>>> So if you are testing, eg. JMS -> JMS and want it
to be really fast
>>>>>>>>> then of course pure JMS to JMS is faster than eg over
Camel as its a
>>>>>>>>> very flexible and transport/protocol agnostic framework.
But
>>>>>>>>> performance improvements is on our roadmap in 2.1.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> View this message in context:
>>>>>>>>>>
>>>>>>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22291841.html
>>>>>>>>>> Sent from the Camel - Users mailing list archive
at Nabble.com.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Claus Ibsen
>>>>>>>>> Apache Camel Committer
>>>>>>>>>
>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>>>>>> View this message in context:
>>>>>>>> http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22292939.html
>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Claus Ibsen
>>>>>> Apache Camel Committer
>>>>>>
>>>>>> Open Source Integration: http://fusesource.com
>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>> --
>>> View this message in context: http://www.nabble.com/Performance-and-MessageSupport.getBody-%281.6.0%29-tp22291841p22310649.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/

Mime
View raw message