maven-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Ashitkin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (MNG-5750) Sporadic failures in concurrent build
Date Sat, 17 Oct 2015 22:50:05 GMT

    [ https://issues.apache.org/jira/browse/MNG-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14962106#comment-14962106
] 

Alexander Ashitkin edited comment on MNG-5750 at 10/17/15 10:49 PM:
--------------------------------------------------------------------

Could you please share your vision on following place (org/apache/maven/lifecycle/internal/BuildListCalculator.java:60):
{code}
           for ( MavenProject project : projects )
            {
                BuilderCommon.attachToThread( project );
                MavenSession copiedSession = session.clone(); // suspicious place
                List<MavenProject> copiedSessionProjects = copiedSession.getProjects();
{code}
Could be here an intention for deep copy? Because debug shows althougs the session is being
cloned, all created sessions share original projects. if some session mutates somehow project
- it actually does it globally. that could potentially result in the issue and explains why
my fix worked.

I modified code like this:
{code}
            for ( MavenProject project : projects )
            {
                BuilderCommon.attachToThread( project ); // Not totally sure if this is needed
for anything
                MavenSession copiedSession = session.clone();
                List<MavenProject> copiedSessionProjects = copiedSession.getProjects();

                for ( int i = 0; i < copiedSessionProjects.size(); i++ )
                {
                    MavenProject mavenProject = copiedSessionProjects.get( i );
                    MavenProject original = session.getProjects().get( i );
                    System.err.println( "Cloned session: " + System.identityHashCode( copiedSession
)
                            + ", project "
                            + mavenProject.getArtifactId() + " shared: " + ( mavenProject
== original ) );
                }
{code}
which gives following output on maven project itself:
{code}
c:\dev\maven-project>mvn clean instal-T 8 -q
Cloned session: 234740890, project maven shared: true
Cloned session: 234740890, project maven-modeshared: true
Cloned session: 234740890, project maven-artifact shared: true
Cloned session: 234740890, project maven-plugin-apshared: true
Cloned session: 234740890, project maven-builder-support shared: true
Cloned session: 234740890, project maven-model-builder shared: true
Cloned session: 234740890, project maven-settingshared: true
Cloned session: 234740890, project maven-settings-builder shared: true
Cloned session: 234740890, project maven-repository-metadatshared: true
Cloned session: 234740890, project maven-aether-provider shared: true
Cloned session: 234740890, project maven-corshared: true
Cloned session: 234740890, project maven-compat shared: true
Cloned session: 234740890, project maven-embedder shared: true
Cloned session: 234740890, project apache-maven shared: true
Cloned session: 79290250, project maven shared: true
Cloned session: 79290250, project maven-modeshared: true
Cloned session: 79290250, project maven-artifact shared: true
Cloned session: 79290250, project maven-plugin-apshared: true
Cloned session: 79290250, project maven-builder-support shared: true
Cloned session: 79290250, project maven-model-builder shared: true
Cloned session: 79290250, project maven-settingshared: true
Cloned session: 79290250, project maven-settings-builder shared: true
Cloned session: 79290250, project maven-repository-metadatshared: true
Cloned session: 79290250, project maven-aether-provider shared: true
Cloned session: 79290250, project maven-corshared: true
Cloned session: 79290250, project maven-compat shared: true
Cloned session: 79290250, project maven-embedder shared: true
Cloned session: 79290250, project apache-maven shared: true
Cloned session: 1586151649, project maven shared: true
Cloned session: 1586151649, project maven-modeshared: true
Cloned session: 1586151649, project maven-artifact shared: true
Cloned session: 1586151649, project maven-plugin-apshared: true
Cloned session: 1586151649, project maven-builder-support shared: true
Cloned session: 1586151649, project maven-model-builder shared: true
Cloned session: 1586151649, project maven-settingshared: true
Cloned session: 1586151649, project maven-settings-builder shared: true
Cloned session: 1586151649, project maven-repository-metadatshared: true
Cloned session: 1586151649, project maven-aether-provider shared: true
Cloned session: 1586151649, project maven-corshared: true
Cloned session: 1586151649, project maven-compat shared: true
Cloned session: 1586151649, project maven-embedder shared: true
Cloned session: 1586151649, project apache-maven shared: true
Cloned session: 179294202, project maven shared: true
.....
{code}

please share your thoughts

thanks in advance, Alexander


was (Author: alexander ashitkin):
Could you please share your vision on following place (org/apache/maven/lifecycle/internal/BuildListCalculator.java:60):
{code}
           for ( MavenProject project : projects )
            {
                BuilderCommon.attachToThread( project );
                MavenSession copiedSession = session.clone(); // suspicious place
                List<MavenProject> copiedSessionProjects = copiedSession.getProjects();
{code}
Could be here an intention for deep copy? Because debug shows althougs the session is being
cloned, all created session share original projects. if some session mutates somehow project
- it actually does it globally. that could potentially result in the issue and explains why
my fix worked.

I modified code like this:
{code}
            for ( MavenProject project : projects )
            {
                BuilderCommon.attachToThread( project ); // Not totally sure if this is needed
for anything
                MavenSession copiedSession = session.clone();
                List<MavenProject> copiedSessionProjects = copiedSession.getProjects();

                for ( int i = 0; i < copiedSessionProjects.size(); i++ )
                {
                    MavenProject mavenProject = copiedSessionProjects.get( i );
                    MavenProject original = session.getProjects().get( i );
                    System.err.println( "Cloned session: " + System.identityHashCode( copiedSession
)
                            + ", project "
                            + mavenProject.getArtifactId() + " shared: " + ( mavenProject
== original ) );
                }
{code}
which gives following output on maven project itself:
{code}
c:\dev\maven-project>mvn clean instal-T 8 -q
Cloned session: 234740890, project maven shared: true
Cloned session: 234740890, project maven-modeshared: true
Cloned session: 234740890, project maven-artifact shared: true
Cloned session: 234740890, project maven-plugin-apshared: true
Cloned session: 234740890, project maven-builder-support shared: true
Cloned session: 234740890, project maven-model-builder shared: true
Cloned session: 234740890, project maven-settingshared: true
Cloned session: 234740890, project maven-settings-builder shared: true
Cloned session: 234740890, project maven-repository-metadatshared: true
Cloned session: 234740890, project maven-aether-provider shared: true
Cloned session: 234740890, project maven-corshared: true
Cloned session: 234740890, project maven-compat shared: true
Cloned session: 234740890, project maven-embedder shared: true
Cloned session: 234740890, project apache-maven shared: true
Cloned session: 79290250, project maven shared: true
Cloned session: 79290250, project maven-modeshared: true
Cloned session: 79290250, project maven-artifact shared: true
Cloned session: 79290250, project maven-plugin-apshared: true
Cloned session: 79290250, project maven-builder-support shared: true
Cloned session: 79290250, project maven-model-builder shared: true
Cloned session: 79290250, project maven-settingshared: true
Cloned session: 79290250, project maven-settings-builder shared: true
Cloned session: 79290250, project maven-repository-metadatshared: true
Cloned session: 79290250, project maven-aether-provider shared: true
Cloned session: 79290250, project maven-corshared: true
Cloned session: 79290250, project maven-compat shared: true
Cloned session: 79290250, project maven-embedder shared: true
Cloned session: 79290250, project apache-maven shared: true
Cloned session: 1586151649, project maven shared: true
Cloned session: 1586151649, project maven-modeshared: true
Cloned session: 1586151649, project maven-artifact shared: true
Cloned session: 1586151649, project maven-plugin-apshared: true
Cloned session: 1586151649, project maven-builder-support shared: true
Cloned session: 1586151649, project maven-model-builder shared: true
Cloned session: 1586151649, project maven-settingshared: true
Cloned session: 1586151649, project maven-settings-builder shared: true
Cloned session: 1586151649, project maven-repository-metadatshared: true
Cloned session: 1586151649, project maven-aether-provider shared: true
Cloned session: 1586151649, project maven-corshared: true
Cloned session: 1586151649, project maven-compat shared: true
Cloned session: 1586151649, project maven-embedder shared: true
Cloned session: 1586151649, project apache-maven shared: true
Cloned session: 179294202, project maven shared: true
.....
{code}

please share your thoughts

thanks in advance, Alexander

> Sporadic failures in concurrent build
> -------------------------------------
>
>                 Key: MNG-5750
>                 URL: https://issues.apache.org/jira/browse/MNG-5750
>             Project: Maven
>          Issue Type: Bug
>          Components: General
>    Affects Versions: 3.1.1, 3.2.1, 3.2.2, 3.2.5
>         Environment: SLES 3.0.80-0.7-default SMP x86_64 GNU/Linux Oracle HotSpot JDK
7u25
> windows server 2008 x64 Oracle HotSpot JDK 7u65/8u25
>            Reporter: Alexander Ashitkin
>            Priority: Critical
>
> We have a large project of 300+ modules which regularly fails with different kind of
errors in different places. The issue is reliably reproduced with parallel build and is not
happens in single threaded. The optimal concurrency level for our project ~10 threads. At
this level ~%20 of builds fail. To workaround the issue we reduced concurrency to 4 in development
builds and reverted production build to 1 thread.
> Main point of failures:
> # Surefire ClassNotFound. Reported and investigated in SUREFIRE-1132. Points to a problem
with MavenProject#getArtifacts - empty set unexpectedly returned.
> # Compiler - unexpected failure because of incorrect classpath (literally all dependencies
are not on the classpath), like: {code}
> 20:20:54 [ERROR] /D:/jenkins/work/workspace/..Request.java:[3,30] package ... does not
exist
> 20:20:54 [ERROR] /D:/jenkins/work/workspace/..Request.java:[4,30] package ... does not
exist
> 20:20:54 [ERROR] /D:/jenkins/work/workspace/..Request.java:[8,25] package ... does not
exist
> 20:20:54 [ERROR] /D:/jenkins/work/workspace/..Request.java:[9,21] package org.joda.time
does not exist
> 20:20:54 [ERROR] /D:/jenkins/work/workspace/..Request.java:[11,37] package com.google.common.base
does not exist
> 20:20:54 [ERROR] /D:/jenkins/work/workspace/..Request.java:[11,1] static import only
from classes and interfaces
> 20:20:54 [ERROR] /D:/jenkins/work/workspace/..Request.java:[12,37] package com.google.common.base
does not exist
> 20:20:54 [ERROR] /D:/jenkins/work/workspace/..Request.java:[12,1] static import only
from classes and interfaces
> {code}
> # Jar - unexpected NPE. Reported with stack traces in MJAR-192. (assembly plusgin seems
to be also affected)
> At this point the issue looks like problem with MavenProject#getArtifacts in concurrent
builds.
> To help with the issue im happy to implement or evaluate any custom assembly to trace
this down. Unfortunately i cannot submit my project - it is proprietary.
> Thanks in advance, Alexander



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message