felix-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stuart McCulloch <mccu...@gmail.com>
Subject Re: NullPointerException in maven-bundle-plugin:3.0.1
Date Tue, 02 Feb 2016 16:34:45 GMT
PS. both Guice and the facade have multi-threaded concurrency tests, so once I confirm this
is what’s going on I’ll also look into why this wasn’t picked up by the existing tests

On Tuesday, 2 February 2016 at 16:13, Stuart McCulloch wrote:

> Hi Stephen,
>  
> Thanks for the update.
>  
> The component shouldn’t be escaping to consumers before contextualize has finished,
that sounds like a bug. Note that while it still looks like Plexus from an API perspective,
ever since Maven 3 it’s actually Guice plus a thin facade to support the same API and behaviour
as the old Plexus container. For example the contextualize step (along with other Plexus phases)
is implemented by registering a custom ProvisionListener with Guice, which gets called as
part of injection and is meant to finish before the object can be seen from the perspective
of wherever it’s being injected.
>  
> There have been some changes to Guice master in a related area, so I’ll try the latest
development snapshot just in case that helps - I’ll also confirm what’s happening in the
facade.
>  
> --
> Cheers, Stuart
>  
> On Tuesday, 2 February 2016 at 15:33, Stephen Evanchik wrote:
>  
> > Hi Stuart,
> >  
> > The synchronized let me see what was happening as far as ordering was
> > concerned. It appears that the contextualize() call races with calls to
> > buildDependencyGraph() and that that Container is not performing lifecycle
> > operations on the DefaultDependencyGraphBuilder object before consumers of
> > that object invoke methods.
> >  
> > I think the maven-bundle-plugin needs to have @threadSafe = false since
> > it's not thread safe due its dependencies not being thread safe. I don't
> > know enough about Maven and Plexus to force some sort of explicit ordering
> > in the bean's lifecycle.
> >  
> > However, this patch does seem to fix the underlying issue though it could
> > be subject to a dead lock if all of the build threads wait on the latch.
> > The real solution is for the container to properly follow concurrent
> > lifecycle rules.
> >  
> > ===================================================================
> > ---
> > src/main/java/org/apache/maven/shared/dependency/graph/internal/DefaultDependencyGraphBuilder.java
> > (revision 1726911)
> > +++
> > src/main/java/org/apache/maven/shared/dependency/graph/internal/DefaultDependencyGraphBuilder.java
> > (working copy)
> > @@ -1,5 +1,9 @@
> > package org.apache.maven.shared.dependency.graph.internal;
> >  
> > +import java.util.Collection;
> > +import java.util.concurrent.CountDownLatch;
> > +import java.util.concurrent.TimeUnit;
> > +
> > /*
> > * Licensed to the Apache Software Foundation (ASF) under one
> > * or more contributor license agreements. See the NOTICE file
> > @@ -33,9 +37,6 @@
> > import org.codehaus.plexus.logging.AbstractLogEnabled;
> > import
> > org.codehaus.plexus.personality.plexus.lifecycle.phase.Contextualizable;
> >  
> > -import java.util.Collection;
> > -import java.util.Collections;
> > -
> > /**
> > * Default dependency graph builder that detects current Maven version to
> > delegate to either Maven 2 or Maven 3 specific
> > * code.
> > @@ -52,6 +53,8 @@
> > {
> > protected PlexusContainer container;
> >  
> > + private final CountDownLatch l = new CountDownLatch(1);
> > +
> > /**
> > * Builds a dependency graph.
> > *
> > @@ -81,6 +84,8 @@
> > {
> > try
> > {
> > + l.await(30, TimeUnit.SECONDS);
> > +
> > String hint = isMaven31() ? "maven31" : isMaven2x() ? "maven2"
> > : "maven3";
> >  
> > DependencyGraphBuilder effectiveGraphBuilder =
> > @@ -90,6 +95,9 @@
> >  
> > return effectiveGraphBuilder.buildDependencyGraph( project,
> > filter, reactorProjects );
> > }
> > + catch ( InterruptedException e ) {
> > + throw new DependencyGraphBuilderException( e.getMessage(), e );
> > + }
> > catch ( ComponentLookupException e )
> > {
> > throw new DependencyGraphBuilderException( e.getMessage(), e );
> > @@ -136,5 +144,7 @@
> > throws ContextException
> > {
> > container = (PlexusContainer) context.get(
> > PlexusConstants.PLEXUS_KEY );
> > +
> > + l.countDown();
> > }
> > }
> >  
> >  
> >  
> > Stephen
> >  
> >  
> > On Tue, Feb 2, 2016 at 10:14 AM, Stephen Evanchik <evanchsa@gmail.com (mailto:evanchsa@gmail.com)>
> > wrote:
> >  
> > > Hi Stuart,
> > >  
> > > I think I found the problem. In maven-dependency-tree:{2.1,2.2} the
> > > Contextualizable.contextualize() method is invoked concurrently with the
> > > two buildDependencyGraph() methods. This is why the container field is
> > > null. I simply synchronized the methods against each other.
> > >  
> > > Stephen
> > >  
> > > On Mon, Feb 1, 2016 at 10:35 AM, Stephen Evanchik <evanchsa@gmail.com (mailto:evanchsa@gmail.com)>
> > > wrote:
> > >  
> > > > Hi Stuart,
> > > >  
> > > > I was looking at some parallel builds we ran over the weekend and notice
> > > > some failures. I started to track them down and noticed that there was
a
> > > > very strange error message which seems to indicate that maven-bundle-plugin
> > > > is not thread safe or has a fairly serious bug:
> > > >  
> > > > [ERROR] Failed to execute goal
> > > > org.apache.felix:maven-bundle-plugin:3.0.1:manifest (bundle-manifest)
on
> > > > project PROJECT_A: Execution bundle-manifest of goal
> > > > org.apache.felix:maven-bundle-plugin:3.0.1:manifest failed: A Jar can
only
> > > > accept a valid file or directory: /full/build/path/PROJECT_B/target/classes
> > > >  
> > > > There is a compile dependency between the projects (PROJECT_A depends
on
> > > > PROJECT_B) but it looks like PROJECT_B was not yet built.
> > > >  
> > > > Stephen
> > > >  
> > > >  
> > > >  
> > > > On Thu, Jan 28, 2016 at 8:37 PM, Stephen Evanchik <evanchsa@gmail.com
(mailto:evanchsa@gmail.com)>
> > > > wrote:
> > > >  
> > > > > Hi Stuart,
> > > > >  
> > > > > A final update for tonight: I was able to reproduce the problem with
> > > > > maven-bundle-plugin:3.0.0 but not with maven-bundle-plugin:2.5.4
even
> > > > > though it looks like the DefaultDependencyGraphBuilder is in all
three
> > > > > versions.
> > > > >  
> > > > > Stephen
> > > > >  
> > > > > On Thu, Jan 28, 2016 at 6:18 PM, Stephen Evanchik <evanchsa@gmail.com
(mailto:evanchsa@gmail.com)>
> > > > > wrote:
> > > > >  
> > > > > > Hi Stuart,
> > > > > >  
> > > > > > Thanks for confirming the behavior. I too noticed the redundant
> > > > > > packaging and execution stanzas. I suspect the original developer
that
> > > > > > added this project didn't understand how the bundle extension
worked.
> > > > > >  
> > > > > > When I noticed the redundancy I updated all of the projects
to use the
> > > > > > <execution>/<package> jar pattern hoping that the
problem would go away.
> > > > > > Unfortunately, I saw an improvement but still have build failures.
> > > > > >  
> > > > > > I'm not sure if this is a bug in Maven, maven-dependency-tree
or,
> > > > > > maven-bundle-plugin
> > > > > >  
> > > > > > Stephen
> > > > > >  
> > > > > >  
> > > > > > On Thu, Jan 28, 2016 at 5:43 PM, Stuart McCulloch <mcculls@gmail.com
(mailto:mcculls@gmail.com)>
> > > > > > wrote:
> > > > > >  
> > > > > > > Hi Stephen,
> > > > > > >  
> > > > > > > I've been able to recreate this with Maven 3.3.9 (by cloning
your
> > > > > > > example
> > > > > > > project several times in a multi-module project) - will
investigate
> > > > > > > further
> > > > > > > over the weekend.
> > > > > > >  
> > > > > > > I did notice you're using the 'bundle' packaging, but you
also have an
> > > > > > > explicit execution of the 'bundle' goal which should already
be
> > > > > > > covered by
> > > > > > > the 'bundle' packaging's lifecycle - is there a reason
for this extra
> > > > > > > execution?
> > > > > > > On 28 Jan 2016 18:00, "Stephen Evanchik" <evanchsa@gmail.com
(mailto:evanchsa@gmail.com)> wrote:
> > > > > > >  
> > > > > > > > Hi Stuart,
> > > > > > > >  
> > > > > > > > I am using Maven 3.3.3 please disregard my earlier
email. I did see
> > > > > > > this
> > > > > > > > problem in 3.3.1 so there doesn't seem to be a difference.
I am
> > > > > > >  
> > > > > > > going to
> > > > > > > > try with 3.3.9 but I'm not going to hold out hope
that it will make a
> > > > > > > > difference.
> > > > > > > >  
> > > > > > > > I think the problem is that DefaultDependencyGraphBuilder
does not
> > > > > > > get
> > > > > > > > initialized properly. This only occurs in parallel
builds and is
> > > > > > >  
> > > > > > > strongly
> > > > > > > > associated with relative timing. For example, if I
issue a mvn -T2.0C
> > > > > > > > install I will see the failure >80% of the time
but if I use mvn -X
> > > > > > > >  
> > > > > > >  
> > > > > > > -T2.0C
> > > > > > > > I cannot reproduce the failure at all.
> > > > > > > >  
> > > > > > > > I'm not sure embedding a pom is the right approach
but here's a
> > > > > > > pastbin
> > > > > > > > link to a sample project that fails:
> > > > > > > >  
> > > > > > > > https://paste.apache.org/dpeQ
> > > > > > > >  
> > > > > > > >  
> > > > > > > > Stephen
> > > > > > > >  
> > > > > > > >  
> > > > > > > > On Wed, Jan 27, 2016 at 4:49 AM, Stuart McCulloch
<mcculls@gmail.com (mailto:mcculls@gmail.com)
> > > > > > > >  
> > > > > > > > wrote:
> > > > > > > >  
> > > > > > > > > On Wednesday, 27 January 2016 at 06:03, Stephen
Evanchik wrote:
> > > > > > > > > > Hi everyone,
> > > > > > > > > >  
> > > > > > > > > > I'm having trouble tracking down an intermittent
but frequent
> > > > > > > build
> > > > > > > > > failure
> > > > > > > > > > using the maven-bundle-plugin to wrap non-OSGi
projects. I'm
> > > > > > > > >  
> > > > > > > > >  
> > > > > > > >  
> > > > > > >  
> > > > > > > using
> > > > > > > > Maven
> > > > > > > > > > 3.3.1 and see the following NPE:
> > > > > > > > > >  
> > > > > > > > > > Caused by: java.lang.NullPointerException
> > > > > > > > > > at
> > > > > > > > > >  
> > > > > > > > >  
> > > > > > > >  
> > > > > > > >  
> > > > > > >  
> > > > > > > org.apache.maven.shared.dependency.graph.internal.DefaultDependencyGraphBuilder.buildDependencyGraph(DefaultDependencyGraphBuilder.java:60)
> > > > > > > > > > at
> > > > > > > > >  
> > > > > > > >  
> > > > > > >  
> > > > > > > org.apache.felix.bundleplugin.BundlePlugin.buildDependencyGraph(BundlePlugin.java:334)
> > > > > > > > > > at
> > > > > > > > >  
> > > > > > > >  
> > > > > > >  
> > > > > > > org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java:359)
> > > > > > > > > > at
> > > > > > > > >  
> > > > > > > >  
> > > > > > >  
> > > > > > > org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
> > > > > > > > > > ... 11 more
> > > > > > > > > >  
> > > > > > > > > > whenever I invoke a parallel build (-T2.0C
for example).
> > > > > > > > > >  
> > > > > > > > > > There are many projects that will fail with
this exception. I can
> > > > > > > > > provide a
> > > > > > > > > > fairly simple one if that makes sense.
> > > > > > > > >  
> > > > > > > > >  
> > > > > > > > > a reproducible test project is always helpful
- have you tried a
> > > > > > > more
> > > > > > > > > recent version of Maven like 3.3.3 or 3.3.9 to
see if that helps?
> > > > > > > > > > It looks like the 3.0.1 version is using
> > > > > > > > >  
> > > > > > > > >  
> > > > > > > >  
> > > > > > >  
> > > > > > > maven-dependency-tree-2.1
> > > > > > > > whose
> > > > > > > > > > buildDependency method looks like:
> > > > > > > > > >  
> > > > > > > > > > public DependencyNode buildDependencyGraph(
MavenProject project,
> > > > > > > > > > ArtifactFilter filter )
> > > > > > > > > > throws DependencyGraphBuilderException
> > > > > > > > > > {
> > > > > > > > > > try
> > > > > > > > > > {
> > > > > > > > > > String hint = isMaven31() ? "maven31" :
isMaven2x() ? "maven2"
> > > > > > > > > > : "maven3";
> > > > > > > > > > getLogger().debug( "building " + hint +
" dependency graph for
> > > > > > > > > > " + project.getId() );
> > > > > > > > > >  
> > > > > > > > > > DependencyGraphBuilder effectiveGraphBuilder
=
> > > > > > > > > > (DependencyGraphBuilder) container.lookup(
> > > > > > > > > > DependencyGraphBuilder.class.getCanonicalName(),
hint );
> > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > > where the NPE is on:
> > > > > > > > > >  
> > > > > > > > > > DependencyGraphBuilder effectiveGraphBuilder
=
> > > > > > > > > > (DependencyGraphBuilder) container.lookup(
> > > > > > > > > > DependencyGraphBuilder.class.getCanonicalName(),
hint );
> > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > > I'm not sure why this could NPE as it seems
like the
> > > > > > > > > > Contextualizable.contextualize() is called
successfully for other
> > > > > > > > > >  
> > > > > > > > >  
> > > > > > > > > projects
> > > > > > > > > > in the build.
> > > > > > > > > >  
> > > > > > > > > > Any ideas on how to track this down? I can't
enable debugging
> > > > > > > (mvn -X)
> > > > > > > > > > because that affects the timing just enough
to avoid the issue.
> > > > > > > > > >  
> > > > > > > > > > Thanks,
> > > > > > > > > > Stephen
> > > > > > > > > >  
> > > > > > > > >  
> > > > > > > >  
> > > > > > > >  
> > > > > > > >  
> > > > > > > > --
> > > > > > > > Stephen Evanchik
> > > > > > > > http://stephen.evanchik.com
> > > > > > > >  
> > > > > > >  
> > > > > > >  
> > > > > >  
> > > > > >  
> > > > > >  
> > > > > >  
> > > > > > --
> > > > > > Stephen Evanchik
> > > > > > http://stephen.evanchik.com
> > > > > >  
> > > > >  
> > > > >  
> > > > >  
> > > > >  
> > > > > --
> > > > > Stephen Evanchik
> > > > > http://stephen.evanchik.com
> > > > >  
> > > >  
> > > >  
> > > >  
> > > >  
> > > > --
> > > > Stephen Evanchik
> > > > http://stephen.evanchik.com
> > > >  
> > >  
> > >  
> > >  
> > >  
> > > --
> > > Stephen Evanchik
> > > http://stephen.evanchik.com
> > >  
> >  
> >  
> >  
> >  
> > --  
> > Stephen Evanchik
> > http://stephen.evanchik.com
> >  
> >  
> >  
>  
>  


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message