Mailing-List: contact dev-help@karaf.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@karaf.apache.org
MIME-Version: 1.0
In-Reply-To: <CAKsTGA6RNDVka5SyeTy0JTbaTGdOGpU7dC3g=40Q1MJUfrQ03w@mail.gmail.com>
References: <CAKsTGA58xasVwuoYwTbETQWiOEk4z_naMCN-bcezzVXWiYphiA@mail.gmail.com>
 <cb933328-5a94-63e1-caeb-1482354037b4@nanthrax.net> <CAKsTGA6DzKbWgRHgfY6Apw2GuP-2Vun7cK8nFZGVV2HzabxoEA@mail.gmail.com>
 <CAA66TpqJ3=VNeSiFVN8yhZQNvpi4eonW3y-kjFY3OwkpYJ72yA@mail.gmail.com>
 <CAAdXmho2NhVAOYkGufkGiL6XRH6GrTf6RWwpi-BBe2fp0vJ2-A@mail.gmail.com> <CAKsTGA6RNDVka5SyeTy0JTbaTGdOGpU7dC3g=40Q1MJUfrQ03w@mail.gmail.com>
From: Guillaume Nodet <gnodet@apache.org>
Date: Wed, 1 Feb 2017 12:54:29 +0100
Message-ID: <CAA66Tpq8=qKOHbMM9sdAJ=xewm9cdqkKOzr8UMfbpMhAxFsu7w@mail.gmail.com>
Subject: Re: [DISCUSS] Living on the edge... of version range
To: dev <dev@karaf.apache.org>
Content-Type: multipart/alternative; boundary=001a1145009e41e0d1054776b7ce
archived-at: Wed, 01 Feb 2017 11:54:52 -0000

--001a1145009e41e0d1054776b7ce
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

2017-02-01 12:35 GMT+01:00 =C5=81ukasz Dywicki <luke@code-house.org>:

> @Guillaume you are right on findEntries part, this will break with
> maven execution.
>

What findEntries part are you talking about ?
I was just referring to the fact that a modified version of
MavenRepositorySystemUtil class would be better embedded directly into
pax-url-aether. Else, you'd have to provide 2 alternative ways to use it,
one by using a fragment and the other one by ensuring that the hacked class
is loaded first in the classloader (in case of a simple classloader).
It just seems easier to me to just move that class in the shaded
pax-url-aether and hack it so that the behavior can be easily controlled
using a boolean on the usual pax-url-aether configuration.


>
> @Grzegorz I do not negate that version, metadata or range resolving
> works. It does with some small exceptions (see PAXURL-342). What
> doesn't work is re-use of ranges from maven build into karaf runtime.
> When you use maven <dependency> and aether to build custom assembly
> feature resolver in runtime will not work for certain cases because of
> version edges.
> When you will take a look on these two files:
> https://github.com/splatch/maven-osgi-resolver/blob/
> master/shared/src/test/java/org/code_house/maven/osgi/
> resolver/shared/version/OsgiVersionRangeTest.java
> https://github.com/splatch/maven-osgi-resolver/blob/
> master/test/src/test/java/org/eclipse/aether/util/version/
> GenericVersionRangeTest.java
> and diff between them https://www.diffchecker.com/eyzDSSXU you will
> find quite big area for troubles.
>
> We about to start using ranges in build because we do not want to have
> 3 different minor versions installed without need. Using ranges in
> both places acutally might speed up running integration tests because
> metadata is already fetched and present in local repository. Otherwise
> our integration tests bootstrap takes 2 minutes just to scan remotes
> for new versions (see above pax url issue).
>
> Cheers,
> Lukasz
>
> 2017-02-01 12:02 GMT+01:00 Grzegorz Grzybek <gr.grzybek@gmail.com>:
> > Hello,
> >
> > Current pax-url-aether has already some custom services DInjected into
> > RepositorySystem, like here[1]. Also in Fabric8v1 and in Karaf I did so=
me
> > tricks to implement non-canonical "update releases" scenario[2].
> > So I think adding configuration options for pax-url to modify the way
> > RepositorySystem is configured should not be a problem.
> >
> > What is the ultimate problem you want to solve? Is it (at lowest level)
> the
> > ability to handle the below URLs?:
> >
> > osgi:install mvn:groupId/artifactId/[lowerBound, upperBound)
> >
> > Currently (pax-url 2.5.2) LATEST, RELEASE and SNAPSHOT versions should =
be
> > handled correctly[3]:
> >  - LATEST instructs AetherBasedResolver to fetch group/artifact
> > metadata.xml and pick latest release OR snapshot
> >  - RELEASE instructs AetherBasedResolver to fetch group/artifact
> > metadata.xml and pick latest release
> >  - SNAPSHOT instructs AetherBasedResolver to fetch group/artifact/versi=
on
> > metadata.xml and pick latest snapshot
> >
> > e.g., in Fabric8v1 I added custom org.eclipse.aether.impl.
> MetadataResolver
> > that is able to resolve metadata ("maven-metadata.xml") even in local
> > repositories into which a SNAPSHOT was installed using `mvn clean
> install`
> > - so the metadata is stored in "maven-metadata-local.xml" file - but th=
e
> > repo is used as remote repository (expecting to return
> "maven-metadata.xml"
> > file.
> >
> > Are you using same version ranges in POM and in features.xml? (I don't
> > argue with the fact that version ranges are used at all in POM :).
> >
> > regards
> > Grzegorz
> > =3D=3D=3D
> > [1]:
> > https://github.com/ops4j/org.ops4j.pax.url/blob/master/pax-
> url-aether/src/main/java/org/ops4j/pax/url/mvn/internal/
> AetherBasedResolver.java#L1168-L1169
> > [2]:
> > https://ops4j1.jira.com/browse/PAXURL-322?focusedCommentId=3D37006&page=
=3D
> com.atlassian.jira.plugin.system.issuetabpanels:comment-
> tabpanel#comment-37006
> > [3]: http://ggrzybek.blogspot.com/2016/10/using-maven-with-osgi-
> part-3.html
> >
> > 2017-02-01 11:44 GMT+01:00 Guillaume Nodet <gnodet@apache.org>:
> >
> >> 2017-02-01 11:31 GMT+01:00 =C5=81ukasz Dywicki <luke@code-house.org>:
> >>
> >> > Thanks for your repiles. If we will manage to get pax-url accepting
> >> > different version range resolving than maven default then I think we
> >> > will not have any troubles with features left. What I was thinking
> >> > about is moving my maven-osgi-resolver to karaf tooling and extendin=
g
> >> > pax-url in the way it could pick up version range resolver
> >> > implementation from fragment bundle. This way we could keep current
> >> > behavior which might be used by someone but also let others use end =
to
> >> > end range support. There are more "extension points" built into
> >> > Aerther which gets normally wired by IoC. Since we can't and we do n=
ot
> >> > want to embed yet-another-ioc-tool for low level stuff we would just
> >> > need to make aether's ServiceLocator entries customizable. It is
> >> > simple Map between role and implementation classes thus would not
> >> > require anything more than bundle.findEntries. This way we could als=
o
> >> > solve pax-url troubles with wagon not loaded up properly.
> >> >
> >>
> >> Won't that make things a bit more complicated for the karaf maven
> plugin ?
> >> It does not run in OSGi, so the fragment stuff won't work.  If the
> problem
> >> is the compatibility, it may still be easier to put the code in
> >> pax-url-eather, and only have a flag to turn the version resolver into
> an
> >> OSGi compatible one, so that the default would be unchanged.
> >> I honestly don't mind, I'm just trying to find the best way to handle
> that.
> >>
> >>
> >> > @Guillaume, we don't need to handle RELEASE flag because this part i=
s
> >> > not subject of version range resolution but VersionResolver. This is
> >> > piece of logic we would not (hopefully) need to amend.
> >> >
> >> > If you will take a look on my current implementation there is mixed
> >> logic:
> >> > https://github.com/splatch/maven-osgi-resolver/blob/
> >> > master/compatible/src/main/java/org/code_house/maven/
> >> > osgi/resolver/compatible/CompatibleOsgiVersionRangeResolver.java#L87
> >> > https://github.com/splatch/maven-osgi-resolver/blob/
> >> > master/strict/src/main/java/org/code_house/maven/osgi/
> resolver/strict/
> >> > StrictOsgiVersionRangeResolver.java#L80
> >> >
> >> > First implementation uses Maven ordering of versions meaning it
> >> > preffers releases over snapshots in selected range. Second
> >> > implementation behaves as OSGi, meaning it will ignore snapshot and
> >> > use regular qualifier comparision but more importantly it will also
> >> > accept just 3.4.0 as a range without upper bound.
> >> >
> >> > Best regards,
> >> > Lukasz
> >> >
> >> > 2017-02-01 9:44 GMT+01:00 Jean-Baptiste Onofr=C3=A9 <jb@nanthrax.net=
>:
> >> > > Hi Lukasz,
> >> > >
> >> > > Thanks for your detailed e-mail and I fully agree with you.
> >> > >
> >> > > I guess the first step would be to improve the version range
> support in
> >> > > Maven URL, and after in the feature resolver.
> >> > >
> >> > > Correct ?
> >> > >
> >> > > Regards
> >> > > JB
> >> > >
> >> > >
> >> > > On 02/01/2017 02:05 AM, =C5=81ukasz Dywicki wrote:
> >> > >>
> >> > >> Dear receivers,
> >> > >> I would like to summarize my research and fight to align version
> range
> >> > >> handling in different parts of karaf related projects. As some of
> you
> >> > >> might not know version ranges are working differently depending o=
n
> >> > >> context we are working in. In general most of logic stays the sam=
e
> >> > >> while there are some edge cases which breaks up everything. But
> let me
> >> > >> start from begining.
> >> > >>
> >> > >> Karaf is OSGi related project which keeps very nice integration
> with
> >> > >> maven based repositories thanks to pax-url. Both environments do
> >> > >> support ranges in quite different way, an example of maven range
> >> > >> understanding is described in maven enforcer plugin documentation
> [1].
> >> > >> Reason why ranges are working differently here and there is a mav=
en
> >> > >> snapshot version and understanding of released version. Osgi
> framework
> >> > >> does not distinguish any of these. It has knowledge of major, min=
or
> >> > >> and micro parts of an version and uses them for comparision but t=
he
> >> > >> qualifier is just a text which might be used for sorting artifact=
s
> >> > >> with same number. This means that for Maven 3.0-SNAPSHOT version =
is
> >> > >> lower than 3.0. In maven there is also knowledge of alpha, beta,
> rc,
> >> > >> cr, milestone, ga and sp (service pack) release types [2].
> >> > >>
> >> > >> Now lets come to places which are using or might be using version
> >> > >> ranges in typical Karaf based project:
> >> > >> - OSGi framework for wiring in packages
> >> > >> - pax-url-mvn for installing maven artifacts
> >> > >> - karaf feature core for choosing dependant features
> >> > >> - maven for including dependant artifacts (ie. feature sets/KARs
> etc)
> >> > >> - karaf-maven-plugin for building assemblies
> >> > >>
> >> > >> When any of range definitions is crossing osgi-maven world proble=
ms
> >> > >> starts to happen. For example range such [2.18, 2.19) in maven wi=
ll
> >> > >> accept 2.19.0-SNAPSHOT while in OSGi it will not. This lead to
> >> > >> situations that these two code parts behave completely differentl=
y
> >> > >> (assuming that camel-core feature is just one bundle):
> >> > >> <bundle>mvn:org.apache.camel/camel-core/${camel.version}</bundle>
> >> > >> <feature version=3D"${camel.version}">camel-core</feature>
> >> > >> This will behave like above but not like bundle statement:
> >> > >>
> >> > >> <repository>mvn:org.apache.camel.karaf/features/${camel.
> >> > version}/xml/features</repository>
> >> > >>
> >> > >> There are some attempts to work around that by using versions
> starting
> >> > >> from ie 2.18.1 so version beginning works just fine but still
> there is
> >> > >> problem of range end. To exclude 2.19-SNAPSHOT in maven you must
> use
> >> > >> "2.19.min" which in osgi will acceptversion 2.19.. Obviously ther=
e
> is
> >> > >> also no way to influence 3rd party so they do not release version
> >> > >> 4.1.0 but 4.1.1 just for our environment pleasure.
> >> > >>
> >> > >> For me it's quite big issue because hitting us on daily basis. We
> have
> >> > >> quite few modules (around 400) which are usualy moving together b=
ut
> >> > >> they should be keeping contract/interfaces on micro versions. Thi=
s
> >> > >> inconsistency lives in Karaf and Pax Url since very long time and
> >> > >> current project infrastructure is not ready to changing that. Fro=
m
> >> > >> other hand keeping this inconsistent will lead to ultimate fail
> some
> >> > >> day and users frustration as well (see KARAF-4105 [3]). Worth to
> point
> >> > >> that this issue pointed out brieefly this issue but didn't solve
> cause
> >> > >> but aligned just one place to maven's logic while keeping all
> others
> >> > >> the same.
> >> > >>
> >> > >> I took my chance and managed to get maven understanding osgi
> version
> >> > >> ranges thanks to core extensions mechanism they have [4]. I also
> >> > >> managed to correct shaded aether inside pax-url [5] so it use
> version
> >> > >> ranges in same way as maven. What I completely failed is making a
> >> > >> custom distro built with my pax-url. Since pax-url-mvn is a start=
up
> >> > >> bundle I can't use overrides for changing it's version and I can'=
t
> >> > >> influence its classes using fragment bundle (yet). To get my own
> >> > >> pax-url I would ned to get rid of framework, but then I have to
> copy
> >> > >> bunch of resources. It would be fine for temporary prosthesis but=
 I
> >> > >> can't rely on it forever. I also got into troubles with
> >> > >> karaf-maven-plugin when setting extra dependency with "my own
> aether".
> >> > >>
> >> > >> As you now know - there is lots of troubles with version ranges
> making
> >> > >> their usage in end-to-end build very difficult. I would love to g=
et
> >> > >> this solved as soon as possible in 4.1 without holding current
> >> > >> release. Aligning all these version range handling is definitelly
> >> > >> doable because from Maven/Aether perspective there is an SPI for
> that.
> >> > >> We just need to deliver it our own VersionRangeResolver interface
> [6].
> >> > >> Open question is shall we keep ordering of versions same as maven
> >> > >> breaking up a little osgi range understanding here.
> >> > >>
> >> > >> [1]
> >> > >> http://maven.apache.org/components/enforcer/enforcer-
> >> > rules/versionRanges.html
> >> > >> [2]
> >> > >> https://github.com/eclipse/aether-core/blob/1.0.x/aether-
> >> > util/src/main/java/org/eclipse/aether/util/version/
> >> > GenericVersion.java#L183
> >> > >> [3] https://issues.apache.org/jira/browse/KARAF-4105
> >> > >> [4] http://markmail.org/message/z6x27umabwqhdjvy
> >> > >> [5]
> >> > >> https://github.com/splatch/maven-osgi-resolver/blob/
> >> > master/compatible-pax/pom.xml
> >> > >> [6]
> >> > >> https://github.com/splatch/maven-osgi-resolver/blob/
> >> > master/compatible-locator/src/main/java/org/apache/maven/
> >> > repository/internal/MavenRepositorySystemUtils.java#L78
> >> > >>
> >> > >> Kind regards,
> >> > >> Lukasz
> >> > >> --
> >> > >> Apache Karaf Committer & PMC
> >> > >> Twitter: @ldywicki
> >> > >> Blog: http://dywicki.pl
> >> > >> Code-House - http://code-house.org
> >> > >>
> >> > >
> >> > > --
> >> > > Jean-Baptiste Onofr=C3=A9
> >> > > jbonofre@apache.org
> >> > > http://blog.nanthrax.net
> >> > > Talend - http://www.talend.com
> >> >
> >>
> >>
> >>
> >> --
> >> ------------------------
> >> Guillaume Nodet
> >>
>


--=20
------------------------
Guillaume Nodet

--001a1145009e41e0d1054776b7ce--