maven-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Osipov (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (MNG-6156) Resolving dependencies is not thread-safe
Date Sat, 14 Jan 2017 23:55:26 GMT

    [ https://issues.apache.org/jira/browse/MNG-6156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15822989#comment-15822989
] 

Michael Osipov edited comment on MNG-6156 at 1/14/17 11:55 PM:
---------------------------------------------------------------

Just tried the above. It still fails. It think that this is a medation problem within Maven
Resolver. Though, it does provide a locking mechanism, it does not reliably work:

{noformat}
[INFO] ------------------------------------------------------------------------
[DEBUG] Concurrent download of C:\Users\mosipov\.m2\repository\org\webjars\bower\datatables.net-buttons\1.2.4\datatables.net-buttons-1.2.4.pom
just finished, skipping download
[INFO] Downloaded: https://repo.maven.apache.org/maven2/org/webjars/bower/datatables.net-buttons/1.2.4/datatables.net-buttons-1.2.4.pom
(0 B at 0.0 KB/sec)
[ERROR] Failed to execute goal on project module064: Could not resolve dependencies for project
org.example:module064:jar:1.0.0-snapshot: Failed to collect dependencies at org.webjars.bower:datatables.net-buttons:jar:1.2.4:
Failed to read artifact descriptor for org.webjars.bower:datatables.net-buttons:jar:1.2.4:
Could not transfer artifact org.webjars.bower:datatables.net-buttons:pom:1.2.4 from/to central
(https://repo.maven.apache.org/maven2): C:\Users\mosipov\.m2\repository\org\webjars\bower\datatables.net-buttons\1.2.4\datatables.net-buttons-1.2.4.pom.part.lock
(Zugriff verweigert) -> [Help 1]
[DEBUG] Concurrent download of C:\Users\mosipov\.m2\repository\org\webjars\bower\datatables.net-buttons\1.2.4\datatables.net-buttons-1.2.4.pom
just finished, skipping download
[INFO] Downloaded: https://repo.maven.apache.org/maven2/org/webjars/bower/datatables.net-buttons/1.2.4/datatables.net-buttons-1.2.4.pom
(0 B at 0.0 KB/sec)
[DEBUG] Concurrent download of C:\Users\mosipov\.m2\repository\org\webjars\bower\datatables.net-buttons\1.2.4\datatables.net-buttons-1.2.4.pom
just finished, skipping download
[INFO] Downloaded: https://repo.maven.apache.org/maven2/org/webjars/bower/datatables.net-buttons/1.2.4/datatables.net-buttons-1.2.4.pom
(0 B at 0.0 KB/sec)
[DEBUG] Concurrent download of C:\Users\mosipov\.m2\repository\org\webjars\bower\datatables.net-buttons\1.2.4\datatables.net-buttons-1.2.4.pom
just finished, skipping download
[INFO] Downloaded: https://repo.maven.apache.org/maven2/org/webjars/bower/datatables.net-buttons/1.2.4/datatables.net-buttons-1.2.4.pom
(0 B at 0.0 KB/sec)
[DEBUG] Writing tracking file C:\Users\mosipov\.m2\repository\org\webjars\bower\datatables.net-buttons\1.2.4\datatables.net-buttons-1.2.4.pom.lastUpdated
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal on project
module064: Could not resolve dependencies for project org.example:module064:jar:1.0.0-snapshot:
Failed to collect dependencies at org.webjars.bower:datatables.net-buttons:jar:1.2.4
	at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:221)
	at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies(LifecycleDependencyResolver.java:128)
	at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:245)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:199)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:185)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:181)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
{noformat}

The problem is that my output looks competely different than yours. If seem to have a Nexus
instance problem. If you connect directly to Maven Central from home it will be different.

Please provide a new log file with debug logging enabled.

Notice that there are two issues here:

1. The write to the {{.lastUpdated}} file is not guarded by a {{FileLock}}
2. Incomplete {{FileLocks}} to {{.part.lock}} files

This is defintively a Maven Resolver issue.

Note that I have tried with {{package}} instead of {{dependency:list}}.


was (Author: michael-o):
Just tried the above. It still fails. It think that this is a medation problem within Maven
Resolver. Though, it does provide a locking mechanism, it does not reliably work:

{noformat}
[INFO] ------------------------------------------------------------------------
[DEBUG] Concurrent download of C:\Users\mosipov\.m2\repository\org\webjars\bower\datatables.net-buttons\1.2.4\datatables.net-buttons-1.2.4.pom
just finished, skipping download
[INFO] Downloaded: https://repo.maven.apache.org/maven2/org/webjars/bower/datatables.net-buttons/1.2.4/datatables.net-buttons-1.2.4.pom
(0 B at 0.0 KB/sec)
[ERROR] Failed to execute goal on project module064: Could not resolve dependencies for project
org.example:module064:jar:1.0.0-snapshot: Failed to collect dependencies at org.webjars.bower:datatables.net-buttons:jar:1.2.4:
Failed to read artifact descriptor for org.webjars.bower:datatables.net-buttons:jar:1.2.4:
Could not transfer artifact org.webjars.bower:datatables.net-buttons:pom:1.2.4 from/to central
(https://repo.maven.apache.org/maven2): C:\Users\mosipov\.m2\repository\org\webjars\bower\datatables.net-buttons\1.2.4\datatables.net-buttons-1.2.4.pom.part.lock
(Zugriff verweigert) -> [Help 1]
[DEBUG] Concurrent download of C:\Users\mosipov\.m2\repository\org\webjars\bower\datatables.net-buttons\1.2.4\datatables.net-buttons-1.2.4.pom
just finished, skipping download
[INFO] Downloaded: https://repo.maven.apache.org/maven2/org/webjars/bower/datatables.net-buttons/1.2.4/datatables.net-buttons-1.2.4.pom
(0 B at 0.0 KB/sec)
[DEBUG] Concurrent download of C:\Users\mosipov\.m2\repository\org\webjars\bower\datatables.net-buttons\1.2.4\datatables.net-buttons-1.2.4.pom
just finished, skipping download
[INFO] Downloaded: https://repo.maven.apache.org/maven2/org/webjars/bower/datatables.net-buttons/1.2.4/datatables.net-buttons-1.2.4.pom
(0 B at 0.0 KB/sec)
[DEBUG] Concurrent download of C:\Users\mosipov\.m2\repository\org\webjars\bower\datatables.net-buttons\1.2.4\datatables.net-buttons-1.2.4.pom
just finished, skipping download
[INFO] Downloaded: https://repo.maven.apache.org/maven2/org/webjars/bower/datatables.net-buttons/1.2.4/datatables.net-buttons-1.2.4.pom
(0 B at 0.0 KB/sec)
[DEBUG] Writing tracking file C:\Users\mosipov\.m2\repository\org\webjars\bower\datatables.net-buttons\1.2.4\datatables.net-buttons-1.2.4.pom.lastUpdated
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal on project
module064: Could not resolve dependencies for project org.example:module064:jar:1.0.0-snapshot:
Failed to collect dependencies at org.webjars.bower:datatables.net-buttons:jar:1.2.4
	at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:221)
	at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies(LifecycleDependencyResolver.java:128)
	at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:245)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:199)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:185)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:181)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
{noformat}

The problem is that my output looks competely different than yours. If seem to have a Nexus
instance problem. If you connect directly to Maven Central from home it will be different.

Please provide a new log file with debug logging enabled.

Notice that there are two issues here:

1. The write to the {{.lastUpdated}} file is not guarded by a {{FileLock}}
2. Incomplete {{FileLocks}} to {{.part.lock}} files

This is defintively a Maven Resolver issue.

> Resolving dependencies is not thread-safe
> -----------------------------------------
>
>                 Key: MNG-6156
>                 URL: https://issues.apache.org/jira/browse/MNG-6156
>             Project: Maven
>          Issue Type: Bug
>          Components: Artifacts and Repositories
>    Affects Versions: 3.3.9
>            Reporter: Roland Illig
>         Attachments: dependency-list.txt, wagon-not-threadsafe.zip
>
>
> When building a multi-module project in parallel, Maven updates the local Maven repository
without taking into account that other threads or processes might do the same at the same
time.
> To reproduce:
> 1.  unpack the attached project
> 2.  {{mvn dependency:list -U -B -T100}}
> See the attached {{dependency-list.txt}} for an example output.
> First thing to notice is that the dependency is downloaded 100 times. This is unexpected,
since the Reactor should coordinate all the modules.
> Second thing to notice is that the build fails, since a dependency "cannot be found".
This is wrong, since the dependency _can_ be found, it's just not processed properly.
> I suspect the {{legacy.DefaultWagonManager}} to be the cause of this, since the typical
error messages are:
> * {{"Downloaded file does not exist: "}}
> * {{"Error copying temporary file to the final destination: "}}
> * {{"*** CHECKSUM FAILED - RETRYING"}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message