karaf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bengt Rodehav <be...@rodehav.com>
Subject Re: XML unmarshalling performance in Karaf
Date Wed, 19 Mar 2014 14:27:29 GMT
OK - thanks.

I would suggest reconsidering the default timeout and set it to 0. Perhaps
coupling it with documentation on what might happen during container
startup.

Also, it seems kind of harsh to incur this overhead on every lookup when
all that is needed is to delay lookup during container startup. Once the
container is started and all bundles are resolved no timeout should be
needed. Perhaps it's better to also set a start level limit so that when
the start level has passed a certain limit, the timeout is set to 0.

/Bengt



2014-03-19 14:33 GMT+01:00 Guillaume Nodet <gnodet@apache.org>:

>
>
>
> 2014-03-19 14:31 GMT+01:00 Bengt Rodehav <bengt@rodehav.com>:
>
> Thanks for the thorough explanation Guillaume.
>>
>> Given that the lookup of the SAXParserFactory is being done "under the
>> hood" everytime I unmarshall an XML file the overhead can become huge. I
>> think it's different if the application itself creates a factory and caches
>> it. A 100 ms timeout on every unmarshall is of course not feasible for a
>> high throughput application.
>>
>> I think I will try setting this property to 0 and try to verify if we run
>> into any problems.
>>
>> I wonder, if I do want to deploy my own implementation of the spec. What
>> then can happen? If I set the timeout to 0 I guess the JRE implementation
>> will be used until my bundle has been resolved. But will my bundle be used
>> once it has been resolved? This would mean that there is a short time at
>> container startup that the JRE version will be used until the proper one is
>> resolved.
>>
>
> Yes, if timeout is set to 0, the JRE version will be used until an
> implementation bundle is resolved.
>
>
>>
>>
>> /Bengt
>>
>>
>>
>>
>>
>>
>>
>>
>> 2014-03-19 14:00 GMT+01:00 Guillaume Nodet <gnodet@apache.org>:
>>
>> Mmh, i would have liked to get away with the explanation ;-)
>>>
>>> So the problem is related to java specifications, like jaxp, stax,
>>> jaxws, jaxb, etc...
>>> The packages are usually provided by the JRE and the discovery is done
>>> using META-INF/services usually.
>>> Unfortunately, this does not work well in OSGi because the application
>>> classloader is used for discovery.
>>>
>>> The servicemix-specs project makes this integration of java specs work
>>> in OSGi.
>>> Over the years, it has improved in various ways, but now, we have the
>>> following:
>>>   * the discovery part of the various specs is enhanced to be OSGi
>>> friendly
>>>   * those enhanced specs are in the ${karaf.home}/endorsed folder so
>>> that they are used instead of the JRE ones
>>>   * the discovery mechanism will look for an implementation bundle in
>>> OSGi, then default to the JRE implementation
>>>
>>> Historically, the discovery of JRE implementations was not possible, so
>>> implementations had to be deployed as bundles.
>>> However, given there's no way to order the resolution of bundles
>>> sufficiently, we needed a timeout so that when a bundle was using one of
>>> the spec, for example the xml parser, the discovery would wait for an
>>> implementation bundle to become available.  Without that timeout, the
>>> discovery could fail during the karaf startup.
>>>
>>> However, since the JRE implementations can now be leveraged (mostly
>>> because the specs are now endorsed instead of being deployed as bundles),
>>> that timeout can be safely set to 0 so that the specs won't wait until a
>>> bundle implementation is available, but will delegate to the JRE directly
>>> if no bundle implementation is present.
>>>
>>> Some weeks ago, I had a quick chat with Jean-Baptiste about setting back
>>> the timeout to 0, but we delayed it for some reason I don't recall.  Maybe
>>> JB remembers ...
>>>
>>>
>>> 2014-03-19 13:28 GMT+01:00 Bengt Rodehav <bengt@rodehav.com>:
>>>
>>> Hello Guillaume!
>>>>
>>>> That made all the difference in the world. The CPU now finally started
>>>> to work and processing time for 1000 of my exchanges went down to 9s from
>>>> 38s.
>>>>
>>>>  But you've got some explaining to do :-)
>>>>
>>>> What is the purpose of this property and why is it set as high as 100
>>>> ms? Also, what can go wrong if I set it to 0?
>>>>
>>>> /Bengt
>>>>
>>>>
>>>> 2014-03-19 11:43 GMT+01:00 Guillaume Nodet <gnodet@apache.org>:
>>>>
>>>> I don't think your problem is concurrency, but just wait.
>>>>> Make the org.apache.servicemix.specs.timeout property is set to 0 in
>>>>> etc/system.properties and retry.
>>>>>
>>>>>
>>>>> 2014-03-19 10:25 GMT+01:00 Bengt Rodehav <bengt@rodehav.com>:
>>>>>
>>>>> To clarify, again looking at the sampler info, the locate() method
>>>>>> spends all its time waiting which indicates a concurrency/synchronization
>>>>>> problem.
>>>>>>
>>>>>> When googling about this I found the following JIRA:
>>>>>>
>>>>>> https://issues.apache.org/jira/browse/SMX4-803
>>>>>>
>>>>>> This seems to have been fixed though.
>>>>>>
>>>>>> I'm not exactly sure how the locator works and if I install a locator
>>>>>> myself or not. I've tried to see what bundle exports
>>>>>> the org.apache.servicemix.specs.locator package but no bundle does
that. So
>>>>>> I guess it's a private package in some bundle.
>>>>>>
>>>>>> Perhaps someone can inform me how this works so I can check if I
need
>>>>>> an updated version of some bundle?
>>>>>>
>>>>>> /Bengt
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2014-03-19 9:56 GMT+01:00 Bengt Rodehav <bengt@rodehav.com>:
>>>>>>
>>>>>> Thanks to Guillaume I managed to get the VisualVM to work with Karaf.
>>>>>>>
>>>>>>> I then ran my transformations a number of times while sampling
to
>>>>>>> see where the time is spent. I attach an image from a snapshot
from
>>>>>>> VisualVM. I'm not sure if it will be accepted by the mailing
list but if
>>>>>>> anyone wants to look at it I can send it to your email directly
if you want.
>>>>>>>
>>>>>>> I'm not an expert on interpreting VisualVM profiling info but
it
>>>>>>> seems to me that in the thread doing the transformation, more
than 90% of
>>>>>>> the time is spent in
>>>>>>> org.apache.servicemix.specs.locator.OsgiLocator.locate(). This
explains why
>>>>>>> the XML unmarshalling runs so much slower in Karaf than outside
of OSGi.
>>>>>>> Can anyone verify if I'have interpreted this correct? If so,
then we
>>>>>>> have a serious performance problem in the OsgiLocator.
>>>>>>>
>>>>>>> /Bengt
>>>>>>>
>>>>>>>
>>>>>>> [image: Infogad bild 1]
>>>>>>>
>>>>>>>
>>>>>>> 2014-03-18 15:28 GMT+01:00 Bengt Rodehav <bengt@rodehav.com>:
>>>>>>>
>>>>>>> I'm using karaf 2.3.4 and Camel 2.13.3.
>>>>>>>>
>>>>>>>> I've been investigating performance problems with Camel's
"sjms"
>>>>>>>> component. Here is the discussion:
>>>>>>>>
>>>>>>>>
>>>>>>>> http://camel.465427.n5.nabble.com/sjms-and-multiple-threads-td5748836.html
>>>>>>>>
>>>>>>>> However, at the end I discovered that my real problem was
the
>>>>>>>> unmarshalling of an XML file in Karaf. For some reason, if
I unmarshall a
>>>>>>>> certain XML file it takes about 105 ms in Karaf. If I do
the same from my
>>>>>>>> Junit test in Eclipse it takes around 10 ms. In fact, in
Eclipse it starts
>>>>>>>> with around 30 ms but consecutive calls gradually go down
to 7-8 ms. In
>>>>>>>> Karaf it doesn't matter how many times I do the unmarshalling.
It stays at
>>>>>>>> about 105 ms everytime.
>>>>>>>>
>>>>>>>> I'm very confused about this.
>>>>>>>>
>>>>>>>> The actual code looks like this (approximately):
>>>>>>>>
>>>>>>>>   public MmlMessage unmarshallMmlMessage(String theXml) throws
>>>>>>>> JAXBException {
>>>>>>>>     final Unmarshaller unMarshaller =
>>>>>>>> cMmlMessageJAXBcontext.createUnmarshaller();
>>>>>>>>     StreamSource ss = new StreamSource(new StringReader(theXml));
>>>>>>>>     long t0 = System.nanoTime();
>>>>>>>>     JAXBElement<?> mmlMessageElement = unMarshaller.unmarshal(ss,
>>>>>>>> MmlMessage.class);
>>>>>>>>     long t1 = System.nanoTime();
>>>>>>>>     MmlMessage mmlMessage = (MmlMessage)
>>>>>>>> mmlMessageElement.getValue();
>>>>>>>>     System.out.println("t1: " + (t1-t0) + " ns");
>>>>>>>>     return mmlMessage;
>>>>>>>>   }
>>>>>>>>
>>>>>>>> The MmlMessage class is generated from an XML schema
>>>>>>>> using maven-jaxb2-plugin. But it shouldn't matter since the
same classes
>>>>>>>> are used within Karaf as outside of Karaf.
>>>>>>>>
>>>>>>>> I assumed that for some reason I'm not running the same code
in
>>>>>>>> Karaf as outside Karaf. When logging the actual class of
the unMarshaller
>>>>>>>> variable I get: "com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl"
>>>>>>>> both within and outside Karaf.
>>>>>>>>
>>>>>>>> The classloader for the unMarshaller in Karaf is:
>>>>>>>> "org.apache.felix.framework.BundleWiringImpl@5740eacd".
>>>>>>>>
>>>>>>>> I thought I had the answer when I noted that outside Karaf
I use
>>>>>>>> the Jaxb implementation that is listed in Camel-jaxb dependencies.
This is
>>>>>>>> version 2.1.13. In Karaf I had installed a jaxb version from
servicemix
>>>>>>>> bundles namely:
>>>>>>>>
>>>>>>>>       <groupId>org.apache.servicemix.bundles</groupId>
>>>>>>>>
>>>>>>>> <artifactId>org.apache.servicemix.bundles.jaxb-impl</artifactId>
>>>>>>>>       <version>2.2.1.1_2</version>
>>>>>>>>
>>>>>>>> So I forced my Junit test to use the same servicemix bundles
>>>>>>>> version but it was still equally fast as before. No where
near the 105 ms I
>>>>>>>> get in Karaf.
>>>>>>>>
>>>>>>>> I realize that this probably is not a Karaf problem per se.
But, I
>>>>>>>> know there are probably lots of people on this mailing that
have handled
>>>>>>>> XML a lot in Karaf. Do you have any tips on what to look
at? What could
>>>>>>>> cause this performance problem?
>>>>>>>>
>>>>>>>> /Bengt
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message