maven-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milos Kleint <>
Subject Re: MNG-5896: Speed up builds by parallel POM downloads
Date Sun, 27 Dec 2015 21:56:58 GMT
Nice work! Looking forward to have this integrated in maven codebase.

We've solved the same problem internally by developing a service + maven
extension that downloads the maven pom files in bulk (one or more zips). We
are using bamboo aws elastic agents a lot and on a clean local repository,
this can save a minute or two for large builds (when your nexus proxy is
also placed in aws as well, when connecting to central even more).


On Sat, Dec 26, 2015 at 9:39 PM, Harald Wellmann <>

> When building a project with dependencies not yet available in the local
> repository, I noticed that Maven first downloads the dependency POMs
> sequentially and then proceeds downloading the dependency JARs with up to 5
> threads in parallel, which is not optimal when the POMs are served by a
> high-latency repository manager.
> There wasn't a lot of feedback on my enhancement request [1] or my
> original StackOverflow question [2], so I started digging into the source
> code and ended up with a patch [3].
> The patch only affects Aether, not Maven Core, but since Aether is not the
> top-level project from the end user perspective and doesn't appear to be
> very active, I thought I'd better contact this mailing list first.
> The basic idea of the patch is a clean separation of POM downloading from
> POM processing in DefaultDependencyCollector. Once these steps are
> separated, it is possible to download dependency POMs asynchronously and in
> parallel, while still processing the POM models sequentially to build the
> dependency graph in the correct order.
> Since DefaultDependencyCollector holds a lot of global state and has
> rather long methods, I started by refactoring this class into smaller
> chunks to make the underlying logic more transparent. That's why the patch
> looks a bit large, but essentially it only affects a single original class.
> I did a local build of Maven 3.4.0-SNAPSHOT using Aether 1.1.0-SNAPSHOT
> with my patch, with all tests passing.
> I also ran maven-integration-testing on this patched Maven 3.4.0-SNAPSHOT,
> with no new tests failing. (There is just one test which is broken since a
> recent change on trunk, see [4].)
> Thanks for reading this far - it would be great if someone would take the
> time to look into the issue and the patch, and advise how to go on.
> [1]
> [2]
> [3]
> [4]
> Regards,
> Harald
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message