maven-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andriy (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (MNG-6030) ReactorModelCache do not used effectively after maven version 3.0.5 which cause a large memory footprint
Date Tue, 31 May 2016 11:19:12 GMT

    [ https://issues.apache.org/jira/browse/MNG-6030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307579#comment-15307579
] 

Andriy edited comment on MNG-6030 at 5/31/16 11:18 AM:
-------------------------------------------------------

What i found while my investigation:
I locally added logging in the cache class to calculate cache miss and found the cache in
not used for parent pom.
In our project I made local change for sharing the cache between modules and it cause smaller
footprint.(the same as was in maven 3.0.5)

according to the DefaultProjectBuilder$InterimResult . it contains ModelBuildingResultinside
it

{code}
        ModelBuildingRequest request = getModelBuildingRequest( config );
....
            ModelBuildingResult result = modelBuilder.build( request );
...
            InterimResult interimResult = new InterimResult( pomFile, request, result, listener,
isRoot );
{code}

and each ModelBuildingRequest has separate instance of the cache inside it 

{code}
        request.setModelCache( new ReactorModelCache() );
{code}

so now we has as mane InterimResult as we has modules and because of the not reused caches
also so many parsed parent poms in each cache. 
also we has a large cache miss while build.

reusing ReactorModelCache will cause smaller footprint of the interimResult  and smaller cache
miss result.


was (Author: sandriy):
What i found while my investigation:
I locally added logging in the cache class to calculate cache miss and found the cache in
not used for parent pom.
In our project I made local change for sharing the cache between modules and it cause smaller
footprint.(the same as was in maven 3.0.5)

according to the DefaultProjectBuilder$InterimResult . in contains ModelBuildingResultinside
it

{code}
        ModelBuildingRequest request = getModelBuildingRequest( config );
....
            ModelBuildingResult result = modelBuilder.build( request );
...
            InterimResult interimResult = new InterimResult( pomFile, request, result, listener,
isRoot );
{code}

and each ModelBuildingRequest has separate instance of the cache inside it 

{code}
        request.setModelCache( new ReactorModelCache() );
{code}

so now we has as mane InterimResult as we has modules and because of the not reused caches
also so many parsed parent poms in each cache. 
also we has a large cache miss while build.

reusing ReactorModelCache will cause smaller footprint of the interimResult  and smaller cache
miss result.

> ReactorModelCache do not used effectively after maven version 3.0.5 which cause a large
memory footprint
> --------------------------------------------------------------------------------------------------------
>
>                 Key: MNG-6030
>                 URL: https://issues.apache.org/jira/browse/MNG-6030
>             Project: Maven
>          Issue Type: Improvement
>          Components: Embedding
>    Affects Versions: 3.1.1, 3.2.5, 3.3.3, 3.3.9
>            Reporter: Andriy
>              Labels: performance
>         Attachments: screenshot-maven-3.0.5.png, screenshot-maven-3.3.9-details.png,
screenshot-maven-3.3.9.png
>
>
> after version 3.0.5 we found a very big memory consumption with maven. needed for build
memory grows from 1G to 5G 
> According to current implementation there were change with commit https://git-wip-us.apache.org/repos/asf?p=maven.git;a=commit;h=e778ea67121f17232a7dced3fb4f0a205ffa64af
  
> https://issues.apache.org/jira/browse/MNG-5312
> {code}
> private ModelBuildingRequest getModelBuildingRequest( InternalConfig config )
>     {
> @@ -256,7 +256,7 @@ public class DefaultProjectBuilder
>          request.setUserProperties( configuration.getUserProperties() );
>          request.setBuildStartTime( configuration.getBuildStartTime() );
>          request.setModelResolver( resolver );
> -        request.setModelCache( config.modelCache );
> +        request.setModelCache( new ReactorModelCache() );
>          return request;
>      }
> {code}
> and as result new ReactorModelCache created for each ModelBuildingRequest which is created
new to parse each pom.xml included in the build as module. 
> so if there is a big projects with a lot of modules ModelBuildingRequest is created for
each of them and parse result is not reused between modules.  and if all this modules has
the same parent pom it parsed each time and stored as many times as many ModelBuildingRequest
was created. As parent pom often contains dependencies, dependencyManagement and common information
for all modules it cause a larger memory footprint then was before. In version 3.0.5 and earlier
ReactorModelCache was single and was reused between ModelBuildingRequest.
> in our cause footprint grew from 1g to 5g.
> it will be better to reuse result of parsing pom files between ModelBuildingRequests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message