maven-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Curtis Rueden <ctrue...@wisc.edu>
Subject Re: Preleminary Maven 3.4.0-SNAPSHOT Testing (Take 3)
Date Thu, 18 Aug 2016 15:24:36 GMT
Hi Stephane,

> The problem is that "pom-whiz" extends from "pom-base". You have a
> scenario where a "whiz" project extends from "pom-base" and then
> import "pom-whiz". Said "pom-whiz", via inheritance, "imports"
> "pom-base" again.

Yep, you got it.

> Independently on the issue described in this thread, this setup is
> broken IMO.

If you read the existing documentation on import scope [1], AFAICS, nowhere
does it say that what my projects do is disallowed or broken or discouraged.

In fact, nowhere does it talk about what happens when dependencyManagement
comes in through both the parent and through imports. It does say, however:

"In the example above Z imports the managed dependencies from both X and Y.
However, both X and Y contain dependency a. Here, version 1.1 of a would be
used since X is declared first and a is not declared in Z's
dependencyManagement."

The explicit claim there is that when there are version conflicts between
the same GA in multiple imported POMs, the first imported POM wins. It's
true that the parent is not an imported POM, it is an _inherited_ POM. But
it's declaration comes before the imported POMs. So by similar logic, its
dependency versions were declared first, so they should win.

I'm not arguing that "I am right and you are wrong" here—I'm saying that
the docs make no promises and there is room for doubt here. I do not think
it is a clear-cut situation. And therefore, we cannot simply write off the
breaking change as a "bug-fix".

>  I already said we spent a lot of time in Spring land to tune our BOMs

Sure. And so did I. And so did many other projects, I'm sure.

>  and one rule of thumb was that you should _never ever_ provide
>  dependency management for something that you're not managing
>  yourself. You should let the component responsible for it drive it.

I absolutely agree that in retrospect, it is much cleaner to have separate
BOMs, and that is certainly what my projects will do moving forward.

I am just looking for a solution which will keep everyone's projects
working as intended while also letting us move Maven forward the way it
should. It sounds like Christian's proposal to bump the POM model version
to 4.1.0 may be the way to go. Maybe that is a discussion for the dev list?

Regards,
Curtis

[1]
https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Importing_Dependencies

--
Curtis Rueden
LOCI software architect - http://loci.wisc.edu/software
ImageJ2 lead, Fiji maintainer - http://imagej.net/User:Rueden
Did you know ImageJ has a forum? http://forum.imagej.net/


On Thu, Aug 18, 2016 at 3:04 AM, Stephane Nicoll <stephane.nicoll@gmail.com>
wrote:

> On Tue, Aug 16, 2016 at 11:14 PM, Curtis Rueden <ctrueden@wisc.edu> wrote:
>
> > Hi Stephane,
> >
> > Apologies up front for my long reply here. I divided into sections to
> help
> > break things up.
> >
> >
> > *== Expected behavior? Or a defect? ==*
> >
> > > if you expect the parent to override something you've defined
> > > in the child, that's not the expected behaviour at all.
> >
> > It certainly _has_ been the expected behavior in my community for the
> past
> > 5 years. Here is a simplified rundown:
> >
> > - pom-base is the parent POM for everything, locking down plugin
> versions,
> > and managing common dependency versions.
> > - pom-whiz extends pom-base to manage dependency versions of "whiz"
> > components.
> > - pom-bang extends pom-base to manage dependency versions of "bang"
> > components.
> > - Whiz-based projects extend pom-whiz to gain dependency management of
> all
> > base and whiz components.
> > - Bang-based projects extend pom-bang to gain dependency management of
> all
> > base and bang components.
> > - Hybrid projects extend pom-base, and import both pom-whiz and pom-bang,
> > to gain dependency management of all base, whiz and bang components.
> >
> > In this scenario, because we use "release early, release often" style
> > development where components are released individually, it is untenable
> for
> > the pom-base version to be totally aligned between the most recent
> > pom-base, pom-whiz and pom-bang releases. I.e.: the whiz developers do
> not
> > want to force releases of bang, and vice versa, just to keep all pom-base
> > versions consistent.
> >
>
> OK, after more coffee I finally managed to get my head around your project.
> The problem is that "pom-whiz" extends from "pom-base". You have a scenario
> where a "whiz" project extends from "pom-base" and then import "pom-whiz".
> Said "pom-whiz", via inheritance, "imports" "pom-base" again.
>
> Independently on the issue described in this thread, this setup is broken
> IMO. I already said we spent a lot of time in Spring land to tune our BOMs
> and one rule of thumb was that you should _never ever_ provide dependency
> management for something that you're not managing yourself. You should let
> the component responsible for it drive it.
>
> Let's take an example in Spring land: Spring Boot is the base for
> application development: it brings Spring Framework and dependency
> management for a bunch of third parties. Spring Cloud is built on top of
> Spring Boot.
>
> Initially, the Spring Cloud BOM was (indirectly) importing dependency
> management from some components managed by Spring Boot. Here's the setup:
> you have a Spring Boot project (with the spring boot parent that provides
> you all that's required to build with it and let's say Spring Framework
> 4.2.2). You decide to include some cloud stuff so you import the
> "spring-cloud-dependencies" bom. When you do that, that BOM MUST NOT have
> any opinion about the boot or framework version to use. If you do, cloud is
> basically overriding the decision of the user. Previously, that BOM had
> some dependency management for boot that would bring, say, Spring Framework
> 4.2.1. So by adding cloud to your project, you know override the framework
> version that the base component provides.
>
> I took us a long time to figure this out but we've managed to structure our
> BOM that way and it works quite well. Because we've done so, we're already
> shielded from this problem but we need this in other contexts (typically
> when a service has to force a dependency change for whatever reason).
>
> HTH,
> S.
>
>
>
> >
> > With Maven 3.3, all of the above works in a very nice way. The dependency
> > versions of base components come from pom-base, while the dependency
> > versions of whiz components come from the pom-whiz import, and versions
> of
> > bang components come from pom-bang. Whereas with Maven 3.4, pom-whiz
> > (assuming it is declared before the pom-bang import) wins over the
> pom-base
> > parent for the base component versions.
> >
> > There are a couple of ways to avoid this:
> >
> > - Add an import of pom-base just before pom-whiz and pom-bang, but this
> was
> > previously redundant with the parent declaration.
> >
> > - Split out the dependency version management part of each POM -- i.e.,
> > create a bom-base, bom-whiz and bom-bang. These boms would all be
> > parentless, and manage only their own respective component sets. Then,
> > hybrid projects which want managed versions of both whiz and bang can
> > continue to import them without inheriting a conflicting set of versions
> > for base components.
> >
> > I like the nice separation of concerns which this second solution offers.
> > But it is more components -- additional complexity which might confuse
> new
> > developers.
> >
> >
> > *== Still a bug in property overrides? ==*
> >
> > Setting aside the issue above, there still seems to be a bug in property
> > overrides, as illustrated by my earlier gist:
> >
> >    https://gist.github.com/ctrueden/a49622e77a65437208feb915a887f929
> >
> > Here we see that with Maven 3.4.0, setting the imagej.version property to
> > 2.0.0-rc-49 in the <properties> section of the project POM has no effect
> on
> > the final resolved version of the associated net.imagej:imagej component.
> > Whereas with Maven 3.3.9, setting that property does modify the resolved
> > dependency version.
> >
> > We all agree that this is a bug, right? If this behavior is not changed,
> I
> > expect it will be the source of frequent bug reports once 3.4.0 is
> > released.
> >
> >
> > *== Build reproducibility ==*
> >
> > One of the beautiful things about Maven is that it tries so hard to
> foster
> > reproducible builds, despite the fact that it draws heavily from the
> > Internet as needed when building. Releases are immutable, you can pin all
> > used plugins to specific releases, etc., so that when you build your
> > project five years later, the same thing is produced as was originally.
> But
> > this change in how dependency versions are computed breaks backwards
> > compatibility in the Maven core itself -- something which (as of this
> > writing) cannot be pinned via the POM. I can understand the desire for
> such
> > core changes between major release versions -- 1.x to 2.x was a big
> > overhaul, and 2.x to 3.x sometimes required massaging of POMs -- but this
> > change is happening in a minor version increment.
> >
> > I do understand that SemVer only promises backwards compatibility of
> > intended behavior, not _all_ behavior. But I think this case is a very
> gray
> > area. The old behavior allowed to have a single POM which acts as a
> parent
> > _and_ a BOM -- with the new behavior, this will no longer be practical
> (see
> > above).
> >
> >
> > *== How to avoid this scenario in the future? ==*
> >
> > I can see that I'm fighting a losing battle here. My community can
> > certainly cut new releases of all our components which are tweaked to
> work
> > properly with Maven 3.4.0. But I am very concerned about the precedent
> > here: at any point in the future, complex builds which used to work might
> > stop doing so, even without a major version increment, due to future
> > changes in the logic of core Maven.
> >
> > It would be ideal if in the future (something for Maven 4?), as much of
> > this logic as possible could be pushed out of core and into plugins, so
> > that they can be pinned in the POM, to promote better build
> > reproducibility.
> >
> > If you actually made it through this whole thing: thank you for reading.
> >
> > Regards,
> > Curtis
> >
> > --
> > Curtis Rueden
> > LOCI software architect - http://loci.wisc.edu/software
> > ImageJ2 lead, Fiji maintainer - http://imagej.net/User:Rueden
> > Did you know ImageJ has a forum? http://forum.imagej.net/
> >
> >
> > On Tue, Aug 16, 2016 at 1:12 PM, Stephane Nicoll <
> > stephane.nicoll@gmail.com>
> > wrote:
> >
> > > Hello Curtis,
> > >
> > > I have no opinion on your project (To be honest, I haven't looked in
> > > details yet, quite a large setup) but if you expect the parent to
> > override
> > > something you've defined in the child, that's not the expected
> behaviour
> > at
> > > all. That's still a problem for you though, I am not denying that.
> > >
> > > Of course, if the issue you're having is some sort of different
> > regression,
> > > we should fix it for sure.
> > >
> > > Thanks,
> > > S.
> > >
> > > On Mon, Aug 15, 2016 at 10:16 PM, Curtis Rueden <ctrueden@wisc.edu>
> > wrote:
> > >
> > > > Hi Stephane,
> > > >
> > > > Why can't we have the best of both worlds? Backwards compatibility,
> but
> > > > with a "stop sucking" flag which enables the new better behavior?
> > > >
> > > > As I said previously, unless the previous behavior is preserved, all
> of
> > > my
> > > > communy's existing releases (hundreds of projects, thousands of tags)
> > > will
> > > > no longer build as intended at time of release.
> > > >
> > > > It could be as simple as the required minimum maven version being set
> > to
> > > > 3.4 to trigger the new behavior.
> > > >
> > > > Regards,
> > > > Curtis
> > > >
> > > > On Aug 15, 2016 2:21 PM, "Stephane Nicoll" <
> stephane.nicoll@gmail.com>
> > > > wrote:
> > > >
> > > > > On Sat, Aug 13, 2016 at 12:49 AM, Christian Schulte <cs@schulte.it
> >
> > > > wrote:
> > > > >
> > > > > > Am 08/13/16 um 00:28 schrieb Christian Schulte:
> > > > > > > reviewing things. So current state of this is: "That's
the
> > > behaviour
> > > > > > > requested and tested during commiting to MNG-5971. Cannot
> > override
> > > > > > > properties? Really requested behaviour? Maybe incorrect.
Need
> to
> > > look
> > > > > at
> > > > > > > it again. There was a reason it got implemented the way
it is."
> > > > > >
> > > > > > The current behaviour is on purpose.
> > > > > >
> > > > > > 1. Read POM.
> > > > > > 2. Recursivley read all parent POMs.
> > > > > > 3. Include (import) dependency declarations top-down at each
> level.
> > > > > > 4. Apply inheritance processing.
> > > > > >
> > > > > > There is no way to use an overridden property value when
> importing
> > > the
> > > > > > depdency declarations because the import happens before
> inheritance
> > > > > > processing. Benefit is the imported dependency declarations
will
> be
> > > > > > available to inheritance processing that way.
> > > > > >
> > > > >
> > > > > I agree and I need to draw the attention to the opposite problem
> > (even
> > > if
> > > > > it was already explained here).
> > > > >
> > > > > The spring ecosystem heavily uses BOMs to define the versions for
> > > Spring
> > > > > related modules. We have those for the framework, spring data,
> spring
> > > > boot
> > > > > and cloud. I took us quite some time to get those BOMs right and
> this
> > > > > effort lead to the creation of MNG-5971.
> > > > >
> > > > > For those asking for a revert, please consider that without it, the
> > BOM
> > > > > feature is pretty much useless (in the sense it does not enforce
> > > > anything).
> > > > > When you have a dependency management section in a project, you
> want
> > to
> > > > > make sure that those versions are going to be used in child
> projects
> > > (and
> > > > > you do so by not specifying any version at all). In a given child
> > > project
> > > > > you can deviate from that rule by overriding the dependency
> > management
> > > > for
> > > > > particular module(s). But it wasn't working with a BOM until now.
> > > > >
> > > > > So, something you couldn't do by importing a BOM is actually
> working
> > by
> > > > > copy/pasting the content of the BOM in the project! I understand
> that
> > > > this
> > > > > may feel a regression for those who were relying on the current
> > > behaviour
> > > > > but I think the current status is much more consistent.
> > > > >
> > > > > Cheers,
> > > > > S.
> > > > >
> > > > >
> > > > > >
> > > > > > Regards,
> > > > > > --
> > > > > > Christian
> > > > > >
> > > > > >
> > > > > > ------------------------------------------------------------
> > > ---------
> > > > > > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> > > > > > For additional commands, e-mail: users-help@maven.apache.org
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message