karaf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guillaume Nodet <gno...@apache.org>
Subject Re: XML unmarshalling performance in Karaf
Date Wed, 19 Mar 2014 13:33:46 GMT
2014-03-19 14:31 GMT+01:00 Bengt Rodehav <bengt@rodehav.com>:

> Thanks for the thorough explanation Guillaume.
>
> Given that the lookup of the SAXParserFactory is being done "under the
> hood" everytime I unmarshall an XML file the overhead can become huge. I
> think it's different if the application itself creates a factory and caches
> it. A 100 ms timeout on every unmarshall is of course not feasible for a
> high throughput application.
>
> I think I will try setting this property to 0 and try to verify if we run
> into any problems.
>
> I wonder, if I do want to deploy my own implementation of the spec. What
> then can happen? If I set the timeout to 0 I guess the JRE implementation
> will be used until my bundle has been resolved. But will my bundle be used
> once it has been resolved? This would mean that there is a short time at
> container startup that the JRE version will be used until the proper one is
> resolved.
>

Yes, if timeout is set to 0, the JRE version will be used until an
implementation bundle is resolved.


>
>
> /Bengt
>
>
>
>
>
>
>
>
> 2014-03-19 14:00 GMT+01:00 Guillaume Nodet <gnodet@apache.org>:
>
> Mmh, i would have liked to get away with the explanation ;-)
>>
>> So the problem is related to java specifications, like jaxp, stax, jaxws,
>> jaxb, etc...
>> The packages are usually provided by the JRE and the discovery is done
>> using META-INF/services usually.
>> Unfortunately, this does not work well in OSGi because the application
>> classloader is used for discovery.
>>
>> The servicemix-specs project makes this integration of java specs work in
>> OSGi.
>> Over the years, it has improved in various ways, but now, we have the
>> following:
>>   * the discovery part of the various specs is enhanced to be OSGi
>> friendly
>>   * those enhanced specs are in the ${karaf.home}/endorsed folder so that
>> they are used instead of the JRE ones
>>   * the discovery mechanism will look for an implementation bundle in
>> OSGi, then default to the JRE implementation
>>
>> Historically, the discovery of JRE implementations was not possible, so
>> implementations had to be deployed as bundles.
>> However, given there's no way to order the resolution of bundles
>> sufficiently, we needed a timeout so that when a bundle was using one of
>> the spec, for example the xml parser, the discovery would wait for an
>> implementation bundle to become available.  Without that timeout, the
>> discovery could fail during the karaf startup.
>>
>> However, since the JRE implementations can now be leveraged (mostly
>> because the specs are now endorsed instead of being deployed as bundles),
>> that timeout can be safely set to 0 so that the specs won't wait until a
>> bundle implementation is available, but will delegate to the JRE directly
>> if no bundle implementation is present.
>>
>> Some weeks ago, I had a quick chat with Jean-Baptiste about setting back
>> the timeout to 0, but we delayed it for some reason I don't recall.  Maybe
>> JB remembers ...
>>
>>
>> 2014-03-19 13:28 GMT+01:00 Bengt Rodehav <bengt@rodehav.com>:
>>
>> Hello Guillaume!
>>>
>>> That made all the difference in the world. The CPU now finally started
>>> to work and processing time for 1000 of my exchanges went down to 9s from
>>> 38s.
>>>
>>>  But you've got some explaining to do :-)
>>>
>>> What is the purpose of this property and why is it set as high as 100
>>> ms? Also, what can go wrong if I set it to 0?
>>>
>>> /Bengt
>>>
>>>
>>> 2014-03-19 11:43 GMT+01:00 Guillaume Nodet <gnodet@apache.org>:
>>>
>>> I don't think your problem is concurrency, but just wait.
>>>> Make the org.apache.servicemix.specs.timeout property is set to 0 in
>>>> etc/system.properties and retry.
>>>>
>>>>
>>>> 2014-03-19 10:25 GMT+01:00 Bengt Rodehav <bengt@rodehav.com>:
>>>>
>>>> To clarify, again looking at the sampler info, the locate() method
>>>>> spends all its time waiting which indicates a concurrency/synchronization
>>>>> problem.
>>>>>
>>>>> When googling about this I found the following JIRA:
>>>>>
>>>>> https://issues.apache.org/jira/browse/SMX4-803
>>>>>
>>>>> This seems to have been fixed though.
>>>>>
>>>>> I'm not exactly sure how the locator works and if I install a locator
>>>>> myself or not. I've tried to see what bundle exports
>>>>> the org.apache.servicemix.specs.locator package but no bundle does that.
So
>>>>> I guess it's a private package in some bundle.
>>>>>
>>>>> Perhaps someone can inform me how this works so I can check if I need
>>>>> an updated version of some bundle?
>>>>>
>>>>> /Bengt
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> 2014-03-19 9:56 GMT+01:00 Bengt Rodehav <bengt@rodehav.com>:
>>>>>
>>>>> Thanks to Guillaume I managed to get the VisualVM to work with Karaf.
>>>>>>
>>>>>> I then ran my transformations a number of times while sampling to
see
>>>>>> where the time is spent. I attach an image from a snapshot from VisualVM.
>>>>>> I'm not sure if it will be accepted by the mailing list but if anyone
wants
>>>>>> to look at it I can send it to your email directly if you want.
>>>>>>
>>>>>> I'm not an expert on interpreting VisualVM profiling info but it
>>>>>> seems to me that in the thread doing the transformation, more than
90% of
>>>>>> the time is spent in
>>>>>> org.apache.servicemix.specs.locator.OsgiLocator.locate(). This explains
why
>>>>>> the XML unmarshalling runs so much slower in Karaf than outside of
OSGi.
>>>>>> Can anyone verify if I'have interpreted this correct? If so, then
we
>>>>>> have a serious performance problem in the OsgiLocator.
>>>>>>
>>>>>> /Bengt
>>>>>>
>>>>>>
>>>>>> [image: Infogad bild 1]
>>>>>>
>>>>>>
>>>>>> 2014-03-18 15:28 GMT+01:00 Bengt Rodehav <bengt@rodehav.com>:
>>>>>>
>>>>>> I'm using karaf 2.3.4 and Camel 2.13.3.
>>>>>>>
>>>>>>> I've been investigating performance problems with Camel's "sjms"
>>>>>>> component. Here is the discussion:
>>>>>>>
>>>>>>>
>>>>>>> http://camel.465427.n5.nabble.com/sjms-and-multiple-threads-td5748836.html
>>>>>>>
>>>>>>> However, at the end I discovered that my real problem was the
>>>>>>> unmarshalling of an XML file in Karaf. For some reason, if I
unmarshall a
>>>>>>> certain XML file it takes about 105 ms in Karaf. If I do the
same from my
>>>>>>> Junit test in Eclipse it takes around 10 ms. In fact, in Eclipse
it starts
>>>>>>> with around 30 ms but consecutive calls gradually go down to
7-8 ms. In
>>>>>>> Karaf it doesn't matter how many times I do the unmarshalling.
It stays at
>>>>>>> about 105 ms everytime.
>>>>>>>
>>>>>>> I'm very confused about this.
>>>>>>>
>>>>>>> The actual code looks like this (approximately):
>>>>>>>
>>>>>>>   public MmlMessage unmarshallMmlMessage(String theXml) throws
>>>>>>> JAXBException {
>>>>>>>     final Unmarshaller unMarshaller =
>>>>>>> cMmlMessageJAXBcontext.createUnmarshaller();
>>>>>>>     StreamSource ss = new StreamSource(new StringReader(theXml));
>>>>>>>     long t0 = System.nanoTime();
>>>>>>>     JAXBElement<?> mmlMessageElement = unMarshaller.unmarshal(ss,
>>>>>>> MmlMessage.class);
>>>>>>>     long t1 = System.nanoTime();
>>>>>>>     MmlMessage mmlMessage = (MmlMessage)
>>>>>>> mmlMessageElement.getValue();
>>>>>>>     System.out.println("t1: " + (t1-t0) + " ns");
>>>>>>>     return mmlMessage;
>>>>>>>   }
>>>>>>>
>>>>>>> The MmlMessage class is generated from an XML schema
>>>>>>> using maven-jaxb2-plugin. But it shouldn't matter since the same
classes
>>>>>>> are used within Karaf as outside of Karaf.
>>>>>>>
>>>>>>> I assumed that for some reason I'm not running the same code
in
>>>>>>> Karaf as outside Karaf. When logging the actual class of the
unMarshaller
>>>>>>> variable I get: "com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl"
>>>>>>> both within and outside Karaf.
>>>>>>>
>>>>>>> The classloader for the unMarshaller in Karaf is:
>>>>>>> "org.apache.felix.framework.BundleWiringImpl@5740eacd".
>>>>>>>
>>>>>>> I thought I had the answer when I noted that outside Karaf I
use the
>>>>>>> Jaxb implementation that is listed in Camel-jaxb dependencies.
This is
>>>>>>> version 2.1.13. In Karaf I had installed a jaxb version from
servicemix
>>>>>>> bundles namely:
>>>>>>>
>>>>>>>       <groupId>org.apache.servicemix.bundles</groupId>
>>>>>>>
>>>>>>> <artifactId>org.apache.servicemix.bundles.jaxb-impl</artifactId>
>>>>>>>       <version>2.2.1.1_2</version>
>>>>>>>
>>>>>>> So I forced my Junit test to use the same servicemix bundles
version
>>>>>>> but it was still equally fast as before. No where near the 105
ms I get in
>>>>>>> Karaf.
>>>>>>>
>>>>>>> I realize that this probably is not a Karaf problem per se. But,
I
>>>>>>> know there are probably lots of people on this mailing that have
handled
>>>>>>> XML a lot in Karaf. Do you have any tips on what to look at?
What could
>>>>>>> cause this performance problem?
>>>>>>>
>>>>>>> /Bengt
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message