maven-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Connolly <stephen.alan.conno...@gmail.com>
Subject Re: "supplies" concept proposal (was "provides" could be "proffers")
Date Fri, 20 Jun 2014 15:55:51 GMT
Well I've had this proposal in one form or another on and off for the past
2-3 years. Jason recently stated that he only wants to look at concrete
proposals as the first stab for what/how to extend the pom format... so
here is a very concrete example proposal ;-)


On 20 June 2014 16:50, Paul Benedict <pbenedict@apache.org> wrote:

> Ah, the woes of fragmented jars. That's the bane of EE dependency
> management. Figuring out the superset/subset stuff is indeed terrible. Good
> job, Stephen, to propose a solution for it.
>
>
> Cheers,
> Paul
>
>
> On Fri, Jun 20, 2014 at 10:03 AM, Stephen Connolly <
> stephen.alan.connolly@gmail.com> wrote:
>
> > Well this started out from a need for a more effective <excludes>.
> >
> > The issue with <excludes> is that you have to apply it to each and every
> > dependency... when what you really want to say is "see this dependency
> > here? well that is the same as X, Y and Z so don't pull them in from the
> > transitive dependencies"
> >
> > If you like that is the second use case in my proposal.
> >
> > The first use case is less relevant from the "better excludes" point of
> > view, but I believe it to be more useful.
> >
> > The third use case is for when you don't trust upstream to get things
> > right. Keep in mind that if, as a general principle, you don't trust
> > upstream to express things mostly correct then what the hell are you
> doing
> > trusting them to express their transitive dependencies correctly?
> >
> > Tooling would allow us to validate a pom's claims of equivalence. But
> > ultimately I think we need this concept to deal with te fragmentation of
> > API jars and implementations of APIs that we see taking place in central.
> >
> >
> > On 20 June 2014 15:27, Paul Benedict <pbenedict@apache.org> wrote:
> >
> > > I am having trouble understanding how grouping together artifacts that
> > are
> > > "equivalent" gain me anything in my project building. I am already
> doing
> > > this in my POM today by excluding what's equivalent. Is this proposal
> > > simply about adding semantic information to the POM so it's more
> apparent
> > > what the intent is?
> > >
> > > Furthermore, whoever defines the equivalence needs to be extremely
> > careful.
> > > Not all "equivalent" jars are actually equivalent. Most people have
> > > learned, for example, that the javaee-api:6.0 jar in Maven is all stubs
> > and
> > > can't be used for unit testing; so people go to find EE jars from JBoss
> > or
> > > GlassFish, to get the full functionality. So I am not sure I'd ever
> want
> > > someone telling me what's equivalent during consuming.
> > >
> > >
> > > Cheers,
> > > Paul
> > >
> > >
> > > On Fri, Jun 20, 2014 at 7:51 AM, Stephen Connolly <
> > > stephen.alan.connolly@gmail.com> wrote:
> > >
> > > > "supplies" concept proposal
> > > > ===========================
> > > >
> > > > Introduction
> > > > ------------
> > > >
> > > > The following is a proposal for Maven in a post-modelVersion-4.0.0
> era.
> > > The
> > > > aim of this proposal is to simplify the management of dependency
> trees
> > in
> > > > the decentralised era of artifact production that we find ourselves
> in.
> > > >
> > > > The core issue is that different organisations can produce artifacts
> > that
> > > > may overlap. The easiest example is the servlet-api. If we restrict
> > > > ourselves to version 2.5 of the servlet specification there are
> quite a
> > > few
> > > > artifacts that all deliver the exact same content:
> > > >
> > > > * `jetty:servlet-api:2.5-6.0.2`
> > > > * `org.daisy.libs:servlet-api:2.5.0`
> > > > * `org.mortbay.jetty:servlet-api-2.5:6.1.14`
> > > > *
> `org.jboss.spec.javax.servlet:jboss-servlet-api_2.5_spec:1.0.1.Final`
> > > > * etc
> > > >
> > > > **Note:** this is a generic problem that is not restricted to the
> > > > servlet-api, the servlet-api just provides the example that will be
> > most
> > > > familiar to everyone.
> > > >
> > > > So where these multiple artifacts supplying the equivalent content
> > > becomes
> > > > a problem is when the dependency tree is being calculated. If you
> have
> > > two
> > > > dependencies each declaring transitive dependencies on different
> > > artifacts
> > > > that supply equivalent content, then you end up with two copies of
> the
> > > same
> > > > JAR file in your classpath.
> > > >
> > > > In the case of the servlet-api, the hack most people use is to
> declare
> > > the
> > > > servlet-api with scope `provided` thus preventing it from being
> > > transitive.
> > > > This is, however, a hack. In a more ideal world it would be better to
> > let
> > > > the servlet-api be transitive and only when we get to the WAR module
> > > would
> > > > we declare that a specific servlet-api is to be provided in the
> > > containers
> > > > that the WAR is targets for deployment into.
> > > >
> > > > We can take a second example that does not have the luxury of a *de
> > > facto*
> > > > hack.
> > > >
> > > > * `javax.faces:jsf-api:2.1`
> > > > * `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec:2.1.28.Final`
> > > > * `org.apache.myfaces.core:myfaces-api:2.1.13`
> > > >
> > > > Now in the case of the JSF API, you are supposed to bundle the JSF
> API
> > in
> > > > your WAR file. So if I use three JSF component libraries, I could
> very
> > > well
> > > > end up with three different but equivalent JSF API jars in my WAR
> file.
> > > >
> > > > Ideally what we want is some way of telling Maven that these
> artifacts
> > > are
> > > > equivalent.
> > > >
> > > > Proposal
> > > > --------
> > > >
> > > > Introduce the concept of "supplies" to the project model. The concept
> > > needs
> > > > three changes to the project model:
> > > >
> > > > 1. An explicit top level construct for a project to explicitly
> declare
> > > > up-front artifacts that it knows - at the time the project is being
> > > > authored - to contain equivalent content to at least a subset of the
> > > > project's content. Declarations could include a claim from: `subset`,
> > > > `superset`, `disjoint` and `equivalent` with the default being
> > > `disjoint`.
> > > > 2. An explicit sub-element of the `dependency` construct to allow
> > > consumers
> > > > to *post-facto* declare a specific dependency as supplying equivalent
> > > > content for other dependencies
> > > > 3. An extension to the `dependency/excludes/exclude` construct to
> allow
> > > > consumers to remove claims a dependency makes with respect to
> supplying
> > > > equivalent content
> > > >
> > > > By way of illustration, here are some examples of these constructs
> > mapped
> > > > to a Model Version 4.0.0 like XML schema. As the
> > post-modelVersion-4.0.0
> > > > schema is not yet known, this represents the best way to illustrate
> how
> > > the
> > > > concept will work, but note that this proposal does not suggest a
> > schema
> > > > for this concept.
> > > >
> > > > ### Example 1
> > > >
> > > > This illustrates how we would want, say, the `myfaces-api` project
> > model
> > > to
> > > > look.
> > > >
> > > > ```
> > > > <project>
> > > >   <groupId>org.apache.myfaces.core</groupId>
> > > >   <artifactId>myfaces-api</artifactId>
> > > >   <version>2.1.3</version>
> > > >   ...
> > > >   <supplyManagement>
> > > >     <supplies>
> > > >       <groupId>javax.faces</groupId>
> > > >       <artifactId>jsf-api</artifactId>
> > > >       <version>[2.1,2.2)</version>
> > > >       <claim>superset</claim>
> > > >     <supplies>
> > > >     <supplies>
> > > >       <groupId>org.jboss.spec.javax.faces</groupId>
> > > >       <artifactId>jboss-jsf-api_2.1_spec</artifactId>
> > > >       <claim>equivalent</claim>
> > > >     <supplies>
> > > >   </supplyManagement>
> > > >   ...
> > > > </project>
> > > > ```
> > > >
> > > > This indicates that the `myfaces-api` artifact is intended to be
> > useable
> > > as
> > > > a drop-in replacement for either the `javax.faces:jsf-api` artifact
> > > within
> > > > a bounded range or for any version of the
> > > > `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec` artifact. If you
> > get
> > > a
> > > > supplier conflict in your classpath, then Maven should fail the
> build.
> > > >
> > > > For example if somebody forked `myfaces-api` but did not list
> > > `myfaces-api`
> > > > in the fork's supplyManagement and you end up with both `myfaces-api`
> > and
> > > > `myfaces-fork-api` in your classpath. Maven can detect that there are
> > two
> > > > dependencies that both claim to supply `javax.faces:jsf-api` and fail
> > the
> > > > build, thereby letting the user add either exclusions or additional
> > > > supplies information to one of the artifacts and preventing duplicate
> > > > artifacts on the classpath. The build need not be failed if the
> > supplies
> > > > claims provide a resolution. e.g. if the claim is `equivalent` then
> > that
> > > > implies that there is a 1:1 mapping and hence the artifacts are
> drop-in
> > > > replacements. However where the claim is `superset` we cannot know
> that
> > > the
> > > > extra content in our artifact is the same as the extra content in
> > another
> > > > artifact which has a superset of `javax.faces:jsf-api`.
> > > >
> > > > ### Example 2
> > > >
> > > > This illustrates a JSF component library that is working with the
> > > existing
> > > > JSF APIs
> > > >
> > > > ```
> > > > <project>
> > > >   ...
> > > >   <dependencies>
> > > >     <dependency>
> > > >       <groupId>javax.faces</groupId>
> > > >       <artifactId>jsf-api</artifactId>
> > > >       <version>2.1</version>
> > > >       <supplyManagement>
> > > >         <supplies>
> > > >           <groupId>org.jboss.spec.javax.faces</groupId>
> > > >           <artifactId>jboss-jsf-api_2.1_spec</artifactId>
> > > >           <claim>equivalent</claim>
> > > >         <supplies>
> > > >         <supplies>
> > > >           <groupId>org.apache.myfaces.core</groupId>
> > > >           <artifactId>myfaces-api</artifactId>
> > > >           <version>[2.1,2.2)</version>
> > > >           <claim>equivalent</claim>
> > > >         </supplies>
> > > >       </supplyManagement>
> > > >     <dependency>
> > > >   </dependencies>
> > > >   ...
> > > > </project>
> > > > ```
> > > >
> > > > In this case we are publishing a transitive dependency with
> additional
> > > > supplyManagement injected. Consumers of this project would thus gain
> > the
> > > > benefit of collapsing their transitive dependencies for any of these
> > > three
> > > > artifacts. As all artifacts are declared with `equivalent` claim,
> thus
> > > the
> > > > nearest of those three artifacts to the project will win as per the
> > > > standard dependency resolution rules when dealing with conflicting
> > > version
> > > > requirements in the transitive dependency tree.
> > > >
> > > > ### Example 3
> > > >
> > > > Finally, there is the case where you need to correct an incorrect
> claim
> > > of
> > > > supply
> > > >
> > > >
> > > > ```
> > > > <project>
> > > >   ...
> > > >   <dependencies>
> > > >     <dependency>
> > > >       <groupId>javax.faces</groupId>
> > > >       <artifactId>jsf-api</artifactId>
> > > >       <version>2.1</version>
> > > >       <exclusions>
> > > >         <exclusion>
> > > >           <groupId>org.jboss.spec.javax.faces</groupId>
> > > >           <artifactId>jboss-jsf-api_2.2_spec</artifactId>
> > > >           <scope>supplies</scope>
> > > >         <exclusion>
> > > >       </exclusions>
> > > >     <dependency>
> > > >   </dependencies>
> > > >   ...
> > > > </project>
> > > > ```
> > > >
> > > > This would typically be coupled with adding back in a correct
> supplies
> > > > definition, but we need to allow for people to correct the graph
> after
> > > the
> > > > fact of their dependencies being deployed to the remote repository.
> > > >
> > > > ### Claim conflict resolution
> > > >
> > > > The four classes of claim can be resolved using the following matrix
> > > >
> > > > ```
> > > >
>  +---------------------------------------------------+
> > > >                  |                         A
> |
> > > >
>  +------------+------------+------------+------------+
> > > >                  | subset     | equivalent | superset   | disjoint
> |
> > > >
> +---+------------+------------+------------+------------+------------+
> > > > |   | subset     | conflict   | A wins     | A wins     | conflict
> |
> > > > |
> +------------+------------+------------+------------+------------+
> > > > |   | equivalent | B wins     | A or B     | A wins     | conflict
> |
> > > > | B
> +------------+------------+------------+------------+------------+
> > > > |   | superset   | B wins     | B wins     | conflict   | conflict
> |
> > > > |
> +------------+------------+------------+------------+------------+
> > > > |   | disjoint   | conflict   | conflict   | conflict   | conflict
> |
> > > >
> +---+------------+------------+------------+------------+------------+
> > > > ```
> > > >
> > > > The default unspecified claim is `disjoint` which indicates that some
> > of
> > > > the content is reproduced, but not all and there is additional
> content
> > > > added. With such a claim there will always be conflict and the build
> > > should
> > > > fail until the Project Model is updated to either remove some of the
> > > claims
> > > > or resolve the dependency clash.
> > > >
> > > > The ideal claim is `equivalent` which indicates that the two
> artifacts
> > > are
> > > > bi-directionally substitutable. This does not mean that the contents
> > are
> > > > identical. It does mean that they both deliver on the same contract
> in
> > an
> > > > equivalent fashion.
> > > >
> > > > The `subset` and `superset` claims are for aggregation APIs. So for
> > > example
> > > > the Java EE Web Profile API is a superset of the various spec APIs
> that
> > > > make up the Java EE Web Profile and a subset of the full Java EE
> > > > specification. The use of the `subset` claim should be reserved to
> > those
> > > > cases that are strict subsets. If anything is added that is not in
> the
> > > > supplied artifact then the correct claim is `disjoint`.
> > > >
> > > > ### Validation of supplies claims
> > > >
> > > > We do not want to introduce Java bias with this feature. As a result
> > the
> > > > validation of claims and supplies directives will be left to plugins.
> > For
> > > > the Java case we should probably provide either/both an enforcer rule
> > or
> > > a
> > > > maven hosted plugin to assist in checking JAR projects against the
> > > declared
> > > > supplies declarations, but Maven core should not be concerned with
> > > solving
> > > > the validation problem.
> > > >
> > > > Similarly, while there may be advantages in a more fine grained API
> > > > contract negotiation between dependencies, to bind such a concept
> into
> > > the
> > > > project model would significantly taint the Maven project model with
> > more
> > > > Java-centric concepts. Given that current software development
> > typically
> > > > uses at least two core languages: Java and JavaScript, we should be
> > > aiming
> > > > to reduce Java-centric constructs from Maven rather than increase
> them.
> > > >
> > > > ### Backporting
> > > >
> > > > It will not be possible to fully express this concept in a
> modelVersion
> > > > 4.0.0 project model. Thus if generating 4.0.0 compatible project
> > models,
> > > > the aim should be to fully resolve the dependencies of the project
> > using
> > > > all available information and express that as the transitive
> > > dependencies.
> > > >
> > > > Thus we will not expose the "supplies" information to modelVersion
> > 4.0.0
> > > > parsers but we will expose the end results of that and present the
> > final
> > > > effective flattened dependency tree.
> > > >
> > > > modelVersion 4.0.0 consumers will thus be no worse off than they
> > already
> > > > are and those consumers understanding newer modelVersions can get the
> > > > enhanced tree resolution that they would not get otherwise.
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message