karaf-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinay Shankar (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0
Date Mon, 21 Aug 2017 14:43:00 GMT

    [ https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135240#comment-16135240
] 

Vinay Shankar edited comment on KARAF-5314 at 8/21/17 2:42 PM:
---------------------------------------------------------------

[~chris@die-schneider.net] - My original email to the mailing list. Was not sent on Friday
for some reason. 

_We have a highly complex and huge feature dependency tree due to the number of OSGI bundles
(components).
When we were using 4.0.9, the plugin would take aroung 5min to build the distribution.
We upgraded to karaf 4.1.1 and the build time has increased significantly.
Now that we refactored our feature dependency tree a bit and added more feature files, the
build time takes around 13min (7min with 4.0.9).
I decided to debug and I noticed a performance bottleneck in the Builder class of the profile
module.
I have created a ticket for the same https://issues.apache.org/jira/browse/KARAF-5314.
Looking in the code, there has been a change in 4.1.1 to use Java 8 Streams API to filter
the required features from the Set of all available features. A “for” loop was being used
in 4.0.9.
I also noticed that there was no cache being used. Each time a required feature was found
in the available set of features, it was not being cached (to avoid checking the set of available
features again if the same feature was found later in the dependency tree).
I have implemented the caching and I tested the code against our feature dependency tree by
pointing to the SNAPSHOT version of the plugin I built locally.
The build only takes 40 seconds to complete now. It is a huge improvement from 13min.
Considering that we are a big team and each of us build locally and we also build on Bamboo
servers for our CI and deployment on multiple environments, this will be a real time saver
for us.
I have pushed the fix to my forked repository and created a pull request for the same https://github.com/apache/karaf/pull/337.
I have also attached the patch file here. Please take a look and review.
I was also wondering (if the review is successful and fix merged to master), when the next
release would be?
We would like to have the fix released in 4.1.3._


was (Author: vinayshankar):
[~chris@die-schneider.net] - My original email to the mailing list. Was not sent on Friday
for some reason. 

_
"We have a highly complex and huge feature dependency tree due to the number of OSGI bundles
(components).
When we were using 4.0.9, the plugin would take aroung 5min to build the distribution.
We upgraded to karaf 4.1.1 and the build time has increased significantly.
Now that we refactored our feature dependency tree a bit and added more feature files, the
build time takes around 13min (7min with 4.0.9).
I decided to debug and I noticed a performance bottleneck in the Builder class of the profile
module.
I have created a ticket for the same https://issues.apache.org/jira/browse/KARAF-5314.
Looking in the code, there has been a change in 4.1.1 to use Java 8 Streams API to filter
the required features from the Set of all available features. A “for” loop was being used
in 4.0.9.
I also noticed that there was no cache being used. Each time a required feature was found
in the available set of features, it was not being cached (to avoid checking the set of available
features again if the same feature was found later in the dependency tree).
I have implemented the caching and I tested the code against our feature dependency tree by
pointing to the SNAPSHOT version of the plugin I built locally.
The build only takes 40 seconds to complete now. It is a huge improvement from 13min.
Considering that we are a big team and each of us build locally and we also build on Bamboo
servers for our CI and deployment on multiple environments, this will be a real time saver
for us.
I have pushed the fix to my forked repository and created a pull request for the same https://github.com/apache/karaf/pull/337.
I have also attached the patch file here. Please take a look and review.
I was also wondering (if the review is successful and fix merged to master), when the next
release would be?
We would like to have the fix released in 4.1.3."
_

> The performance of profile builder used by karaf maven plugin has reduced significantly
in 4.1 compared to 4.0
> --------------------------------------------------------------------------------------------------------------
>
>                 Key: KARAF-5314
>                 URL: https://issues.apache.org/jira/browse/KARAF-5314
>             Project: Karaf
>          Issue Type: Bug
>            Reporter: Vinay Shankar
>             Fix For: 4.2.0
>
>
> The performance of profile builder used by karaf maven plugin has reduced significantly
in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features from the
set of all features present in the repositories that are part of the profile. This is done
in the "addFeatures" method in the "Builder.java" class in the "org.apache.karaf.profile.core"
bundle. This change (from 4.0) has drastically reduced the performance. For a profile with
~900 features in all the repositories in the profile and ~300 required features from a highly
complex feature dependency tree, this function is taking around 13min to complete. The same
execution took around 3-5min in 4.0 (where simple for loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message