karaf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Łukasz Dywicki <l...@code-house.org>
Subject Re: [DISCUSS] Living on the edge... of version range
Date Wed, 01 Feb 2017 13:14:18 GMT
What I was thinking about was something like that:
AetherBasedResolver.class.getResources("META-INF/pax-customization.properties")
then just parsing it and pushing into ServiceLocator instance created
by MavenRepositorySystemUtil. We can do that because
DefaultServiceLocator implementation is mutable as Grzegorz pointed
out. Since we will be in internal package getResources should behave
properly in both cases.

Cheers,
Lukasz



2017-02-01 12:54 GMT+01:00 Guillaume Nodet <gnodet@apache.org>:
> 2017-02-01 12:35 GMT+01:00 Łukasz Dywicki <luke@code-house.org>:
>
>> @Guillaume you are right on findEntries part, this will break with
>> maven execution.
>>
>
> What findEntries part are you talking about ?
> I was just referring to the fact that a modified version of
> MavenRepositorySystemUtil class would be better embedded directly into
> pax-url-aether. Else, you'd have to provide 2 alternative ways to use it,
> one by using a fragment and the other one by ensuring that the hacked class
> is loaded first in the classloader (in case of a simple classloader).
> It just seems easier to me to just move that class in the shaded
> pax-url-aether and hack it so that the behavior can be easily controlled
> using a boolean on the usual pax-url-aether configuration.
>
>
>
>>
>> @Grzegorz I do not negate that version, metadata or range resolving
>> works. It does with some small exceptions (see PAXURL-342). What
>> doesn't work is re-use of ranges from maven build into karaf runtime.
>> When you use maven <dependency> and aether to build custom assembly
>> feature resolver in runtime will not work for certain cases because of
>> version edges.
>> When you will take a look on these two files:
>> https://github.com/splatch/maven-osgi-resolver/blob/
>> master/shared/src/test/java/org/code_house/maven/osgi/
>> resolver/shared/version/OsgiVersionRangeTest.java
>> https://github.com/splatch/maven-osgi-resolver/blob/
>> master/test/src/test/java/org/eclipse/aether/util/version/
>> GenericVersionRangeTest.java
>> and diff between them https://www.diffchecker.com/eyzDSSXU you will
>> find quite big area for troubles.
>>
>> We about to start using ranges in build because we do not want to have
>> 3 different minor versions installed without need. Using ranges in
>> both places acutally might speed up running integration tests because
>> metadata is already fetched and present in local repository. Otherwise
>> our integration tests bootstrap takes 2 minutes just to scan remotes
>> for new versions (see above pax url issue).
>>
>> Cheers,
>> Lukasz
>>
>> 2017-02-01 12:02 GMT+01:00 Grzegorz Grzybek <gr.grzybek@gmail.com>:
>> > Hello,
>> >
>> > Current pax-url-aether has already some custom services DInjected into
>> > RepositorySystem, like here[1]. Also in Fabric8v1 and in Karaf I did some
>> > tricks to implement non-canonical "update releases" scenario[2].
>> > So I think adding configuration options for pax-url to modify the way
>> > RepositorySystem is configured should not be a problem.
>> >
>> > What is the ultimate problem you want to solve? Is it (at lowest level)
>> the
>> > ability to handle the below URLs?:
>> >
>> > osgi:install mvn:groupId/artifactId/[lowerBound, upperBound)
>> >
>> > Currently (pax-url 2.5.2) LATEST, RELEASE and SNAPSHOT versions should be
>> > handled correctly[3]:
>> >  - LATEST instructs AetherBasedResolver to fetch group/artifact
>> > metadata.xml and pick latest release OR snapshot
>> >  - RELEASE instructs AetherBasedResolver to fetch group/artifact
>> > metadata.xml and pick latest release
>> >  - SNAPSHOT instructs AetherBasedResolver to fetch group/artifact/version
>> > metadata.xml and pick latest snapshot
>> >
>> > e.g., in Fabric8v1 I added custom org.eclipse.aether.impl.
>> MetadataResolver
>> > that is able to resolve metadata ("maven-metadata.xml") even in local
>> > repositories into which a SNAPSHOT was installed using `mvn clean
>> install`
>> > - so the metadata is stored in "maven-metadata-local.xml" file - but the
>> > repo is used as remote repository (expecting to return
>> "maven-metadata.xml"
>> > file.
>> >
>> > Are you using same version ranges in POM and in features.xml? (I don't
>> > argue with the fact that version ranges are used at all in POM :).
>> >
>> > regards
>> > Grzegorz
>> > ===
>> > [1]:
>> > https://github.com/ops4j/org.ops4j.pax.url/blob/master/pax-
>> url-aether/src/main/java/org/ops4j/pax/url/mvn/internal/
>> AetherBasedResolver.java#L1168-L1169
>> > [2]:
>> > https://ops4j1.jira.com/browse/PAXURL-322?focusedCommentId=37006&page=
>> com.atlassian.jira.plugin.system.issuetabpanels:comment-
>> tabpanel#comment-37006
>> > [3]: http://ggrzybek.blogspot.com/2016/10/using-maven-with-osgi-
>> part-3.html
>> >
>> > 2017-02-01 11:44 GMT+01:00 Guillaume Nodet <gnodet@apache.org>:
>> >
>> >> 2017-02-01 11:31 GMT+01:00 Łukasz Dywicki <luke@code-house.org>:
>> >>
>> >> > Thanks for your repiles. If we will manage to get pax-url accepting
>> >> > different version range resolving than maven default then I think we
>> >> > will not have any troubles with features left. What I was thinking
>> >> > about is moving my maven-osgi-resolver to karaf tooling and extending
>> >> > pax-url in the way it could pick up version range resolver
>> >> > implementation from fragment bundle. This way we could keep current
>> >> > behavior which might be used by someone but also let others use end
to
>> >> > end range support. There are more "extension points" built into
>> >> > Aerther which gets normally wired by IoC. Since we can't and we do
not
>> >> > want to embed yet-another-ioc-tool for low level stuff we would just
>> >> > need to make aether's ServiceLocator entries customizable. It is
>> >> > simple Map between role and implementation classes thus would not
>> >> > require anything more than bundle.findEntries. This way we could also
>> >> > solve pax-url troubles with wagon not loaded up properly.
>> >> >
>> >>
>> >> Won't that make things a bit more complicated for the karaf maven
>> plugin ?
>> >> It does not run in OSGi, so the fragment stuff won't work.  If the
>> problem
>> >> is the compatibility, it may still be easier to put the code in
>> >> pax-url-eather, and only have a flag to turn the version resolver into
>> an
>> >> OSGi compatible one, so that the default would be unchanged.
>> >> I honestly don't mind, I'm just trying to find the best way to handle
>> that.
>> >>
>> >>
>> >> > @Guillaume, we don't need to handle RELEASE flag because this part
is
>> >> > not subject of version range resolution but VersionResolver. This is
>> >> > piece of logic we would not (hopefully) need to amend.
>> >> >
>> >> > If you will take a look on my current implementation there is mixed
>> >> logic:
>> >> > https://github.com/splatch/maven-osgi-resolver/blob/
>> >> > master/compatible/src/main/java/org/code_house/maven/
>> >> > osgi/resolver/compatible/CompatibleOsgiVersionRangeResolver.java#L87
>> >> > https://github.com/splatch/maven-osgi-resolver/blob/
>> >> > master/strict/src/main/java/org/code_house/maven/osgi/
>> resolver/strict/
>> >> > StrictOsgiVersionRangeResolver.java#L80
>> >> >
>> >> > First implementation uses Maven ordering of versions meaning it
>> >> > preffers releases over snapshots in selected range. Second
>> >> > implementation behaves as OSGi, meaning it will ignore snapshot and
>> >> > use regular qualifier comparision but more importantly it will also
>> >> > accept just 3.4.0 as a range without upper bound.
>> >> >
>> >> > Best regards,
>> >> > Lukasz
>> >> >
>> >> > 2017-02-01 9:44 GMT+01:00 Jean-Baptiste Onofré <jb@nanthrax.net>:
>> >> > > Hi Lukasz,
>> >> > >
>> >> > > Thanks for your detailed e-mail and I fully agree with you.
>> >> > >
>> >> > > I guess the first step would be to improve the version range
>> support in
>> >> > > Maven URL, and after in the feature resolver.
>> >> > >
>> >> > > Correct ?
>> >> > >
>> >> > > Regards
>> >> > > JB
>> >> > >
>> >> > >
>> >> > > On 02/01/2017 02:05 AM, Łukasz Dywicki wrote:
>> >> > >>
>> >> > >> Dear receivers,
>> >> > >> I would like to summarize my research and fight to align version
>> range
>> >> > >> handling in different parts of karaf related projects. As
some of
>> you
>> >> > >> might not know version ranges are working differently depending
on
>> >> > >> context we are working in. In general most of logic stays
the same
>> >> > >> while there are some edge cases which breaks up everything.
But
>> let me
>> >> > >> start from begining.
>> >> > >>
>> >> > >> Karaf is OSGi related project which keeps very nice integration
>> with
>> >> > >> maven based repositories thanks to pax-url. Both environments
do
>> >> > >> support ranges in quite different way, an example of maven
range
>> >> > >> understanding is described in maven enforcer plugin documentation
>> [1].
>> >> > >> Reason why ranges are working differently here and there is
a maven
>> >> > >> snapshot version and understanding of released version. Osgi
>> framework
>> >> > >> does not distinguish any of these. It has knowledge of major,
minor
>> >> > >> and micro parts of an version and uses them for comparision
but the
>> >> > >> qualifier is just a text which might be used for sorting artifacts
>> >> > >> with same number. This means that for Maven 3.0-SNAPSHOT version
is
>> >> > >> lower than 3.0. In maven there is also knowledge of alpha,
beta,
>> rc,
>> >> > >> cr, milestone, ga and sp (service pack) release types [2].
>> >> > >>
>> >> > >> Now lets come to places which are using or might be using
version
>> >> > >> ranges in typical Karaf based project:
>> >> > >> - OSGi framework for wiring in packages
>> >> > >> - pax-url-mvn for installing maven artifacts
>> >> > >> - karaf feature core for choosing dependant features
>> >> > >> - maven for including dependant artifacts (ie. feature sets/KARs
>> etc)
>> >> > >> - karaf-maven-plugin for building assemblies
>> >> > >>
>> >> > >> When any of range definitions is crossing osgi-maven world
problems
>> >> > >> starts to happen. For example range such [2.18, 2.19) in maven
will
>> >> > >> accept 2.19.0-SNAPSHOT while in OSGi it will not. This lead
to
>> >> > >> situations that these two code parts behave completely differently
>> >> > >> (assuming that camel-core feature is just one bundle):
>> >> > >> <bundle>mvn:org.apache.camel/camel-core/${camel.version}</bundle>
>> >> > >> <feature version="${camel.version}">camel-core</feature>
>> >> > >> This will behave like above but not like bundle statement:
>> >> > >>
>> >> > >> <repository>mvn:org.apache.camel.karaf/features/${camel.
>> >> > version}/xml/features</repository>
>> >> > >>
>> >> > >> There are some attempts to work around that by using versions
>> starting
>> >> > >> from ie 2.18.1 so version beginning works just fine but still
>> there is
>> >> > >> problem of range end. To exclude 2.19-SNAPSHOT in maven you
must
>> use
>> >> > >> "2.19.min" which in osgi will acceptversion 2.19.. Obviously
there
>> is
>> >> > >> also no way to influence 3rd party so they do not release
version
>> >> > >> 4.1.0 but 4.1.1 just for our environment pleasure.
>> >> > >>
>> >> > >> For me it's quite big issue because hitting us on daily basis.
We
>> have
>> >> > >> quite few modules (around 400) which are usualy moving together
but
>> >> > >> they should be keeping contract/interfaces on micro versions.
This
>> >> > >> inconsistency lives in Karaf and Pax Url since very long time
and
>> >> > >> current project infrastructure is not ready to changing that.
From
>> >> > >> other hand keeping this inconsistent will lead to ultimate
fail
>> some
>> >> > >> day and users frustration as well (see KARAF-4105 [3]). Worth
to
>> point
>> >> > >> that this issue pointed out brieefly this issue but didn't
solve
>> cause
>> >> > >> but aligned just one place to maven's logic while keeping
all
>> others
>> >> > >> the same.
>> >> > >>
>> >> > >> I took my chance and managed to get maven understanding osgi
>> version
>> >> > >> ranges thanks to core extensions mechanism they have [4].
I also
>> >> > >> managed to correct shaded aether inside pax-url [5] so it
use
>> version
>> >> > >> ranges in same way as maven. What I completely failed is making
a
>> >> > >> custom distro built with my pax-url. Since pax-url-mvn is
a startup
>> >> > >> bundle I can't use overrides for changing it's version and
I can't
>> >> > >> influence its classes using fragment bundle (yet). To get
my own
>> >> > >> pax-url I would ned to get rid of framework, but then I have
to
>> copy
>> >> > >> bunch of resources. It would be fine for temporary prosthesis
but I
>> >> > >> can't rely on it forever. I also got into troubles with
>> >> > >> karaf-maven-plugin when setting extra dependency with "my
own
>> aether".
>> >> > >>
>> >> > >> As you now know - there is lots of troubles with version ranges
>> making
>> >> > >> their usage in end-to-end build very difficult. I would love
to get
>> >> > >> this solved as soon as possible in 4.1 without holding current
>> >> > >> release. Aligning all these version range handling is definitelly
>> >> > >> doable because from Maven/Aether perspective there is an SPI
for
>> that.
>> >> > >> We just need to deliver it our own VersionRangeResolver interface
>> [6].
>> >> > >> Open question is shall we keep ordering of versions same as
maven
>> >> > >> breaking up a little osgi range understanding here.
>> >> > >>
>> >> > >> [1]
>> >> > >> http://maven.apache.org/components/enforcer/enforcer-
>> >> > rules/versionRanges.html
>> >> > >> [2]
>> >> > >> https://github.com/eclipse/aether-core/blob/1.0.x/aether-
>> >> > util/src/main/java/org/eclipse/aether/util/version/
>> >> > GenericVersion.java#L183
>> >> > >> [3] https://issues.apache.org/jira/browse/KARAF-4105
>> >> > >> [4] http://markmail.org/message/z6x27umabwqhdjvy
>> >> > >> [5]
>> >> > >> https://github.com/splatch/maven-osgi-resolver/blob/
>> >> > master/compatible-pax/pom.xml
>> >> > >> [6]
>> >> > >> https://github.com/splatch/maven-osgi-resolver/blob/
>> >> > master/compatible-locator/src/main/java/org/apache/maven/
>> >> > repository/internal/MavenRepositorySystemUtils.java#L78
>> >> > >>
>> >> > >> Kind regards,
>> >> > >> Lukasz
>> >> > >> --
>> >> > >> Apache Karaf Committer & PMC
>> >> > >> Twitter: @ldywicki
>> >> > >> Blog: http://dywicki.pl
>> >> > >> Code-House - http://code-house.org
>> >> > >>
>> >> > >
>> >> > > --
>> >> > > Jean-Baptiste Onofré
>> >> > > jbonofre@apache.org
>> >> > > http://blog.nanthrax.net
>> >> > > Talend - http://www.talend.com
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> ------------------------
>> >> Guillaume Nodet
>> >>
>>
>
>
>
> --
> ------------------------
> Guillaume Nodet

Mime
View raw message