ant-ivy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Loren Kratzke <LKrat...@blueorigin.com>
Subject RE: cache busting and integration question
Date Fri, 03 Apr 2015 15:23:13 GMT
(I apologize for cross posting this to the users and developers lists but it seemed like the
right thing to do to spread this info.)



So in the end technically there was no bug however the default behavior was different than
what I was expecting. I agree about preserving default behavior but there might be some things
that could be done to improve or augment default behavior.



In the meantime, here is a list of settings that need to be altered from their default to
provide the behavior that I needed. These settings will give you the Maven experience. This
I believe is a very important experience that should be easily attained. I had to debug to
figure this out, but in the end Ivy was totally able to provide the desired behavior and still
be Ivy.



1. Use dedicated caches per resolver (or at the very least a dedicated cache for the local
filesystem resolver). Otherwise, Ivy will not distinguish between local and remote artifacts
in the cache and may arbitrarily substitute one for the other.



2. Set defaultLatestStrategy="latest.time" in the settings element. Otherwise Ivy will use
only alphanumeric sort when selecting between artifacts of the same version in different repositories.
This is not useful at all when trying to pick up the latest integration build. The result
is that you will get the first one found or the last one found based upon other configuration
elements.



3. Set the property ivy.resolver.default.check.modified="true" so that Ivy does not falsely
assume that an artifact never changes.



4. Set changingPattern=".*" on all resolvers so that Ivy will not exclude certain (by default
all) revisions from being checked for modifications. In other words, the checkModified setting
has no effect until you provide a pattern that matches the version number of the artifacts
that are being resolved.



5. Use the force="true" attribute on a resolver if you wish to have it considered even after
an artifact has been located by a previous resolver in the chain. This is fine tuning behavior
for a build and the default is ok, but the behavior must be understood clearly as documented
here. Generally speaking, without using this attribute (or setting it to false), and assuming
that you use your internal resolver to publish local dev builds, listing your internal resolver
first in your chain will always get you a locally published artifact even if there is a newer
one in the Nexus repository. Setting this to true on a subsequent Nexus (url) resolver will
select the newer of your local artifact (if present) and the Nexus artifact (if present).
Not using this when publishing to your local repository means that you will have to delete
your local repository when you want to pull an artifact from Nexus again. The opposite holds
true - if you set force="true" on the Nexus (url) repository then you do not need to torch
your local repository to get newer artifacts, but you may get newer artifacts when you least
expect it if something somewhere else is publishing to Nexus.



The last item (force attribute) is a very useful feature once you understand what how it behaves.
What it means is, "force the artifact found by this resolver to be compared using the latestStretegy,
even if a previous resolver has found an artifact of this name and version". Just keep in
mind that not setting changing pattern and/or not setting checkModified are enough to abort
the process such that subsequent artifact even if found, will not be considered.



I really hope this helps users coming from the Maven world understand how to get the Maven
experience from Ivy. And with that out of the way, users can then begin to leverage other
Ivy features. Ivy provides the ability to do things that Maven just can't do, and with language
besides just Java. That is huge and guarantees that Ivy will be around for a very long time.



L.K.


From: Loren Kratzke
Sent: Friday, March 20, 2015 3:12 PM
To: 'ivy-user@ant.apache.org'
Subject: cache busting and integration question

I am encountering challenges with the Ivy cache and also with setting up what I would consider
to be a typical (if not classic) developer workflow. Here is the desired workflow:

I have a Nexus repository (releases and snapshots) plus a local file system repository used
for local development. I have Project which depends upon Library. I wish to modify Library,
publish locally, and then pull it into the local build of Project.

And for my Jenkins "continuous" integration server, if the copy of Library-1.0.0-SNAPSHOT
in Nexus is newer than what is in the cache on the build machine, then I would like Ivy to
grab the newer version (just like Maven does).

ISSUE #1

Assuming I start with an empty cache and empty internal repo on my local dev machine, I build
Project and it pulls Library from Nexus. Perfect. Works great. But then if I publish Library
locally and build Project again, I will get the cached version of Library that came from Nexus.
No matter what I do, I will always get the cached copy. This happens even though my chain
resolver specifies my internal repo before the Nexus repo.

If I blow away the cache, then I get my locally published build, but only if I blow away the
cache. Otherwise I get stale stuff. Issue #2 below is all about stale stuff.

I debugged Ivy (for hours upon hours) and during the resolve, it checks my internal repo,
finds the fresh artifact (yay!) and the next thing it does is checks its cache, finds a module
descriptor that points at Nexus, and proceeds to pull in the wrong artifact into Project.

The behavior is either a bug, or implies that Ivy assumes that an artifact can only come from
one place, and if it came from there once, it will come from there forever, and will never
change. Again, Ivy finds the artifact in local repo, stops searching, then ends up delivering
the cached artifact that came from Nexus sometime earlier.

ISSUE #2

I have been working on this problem for quite some time and I thought I had it fixed, but
I don't. That is, my integration server needs the latest snapshot build of Library-1.0.0-SNAPSHOT.
But what I get is the cached version from the Ivy cache. An SVN commit happens, triggers a
build of Library, Library get published, triggers a build of Project, and a stale build of
Library is delivered to Project from the cache (which breaks Project build).

SUMMARY

Ivy will resolve and cache deps, but once a dep is in the cache, that's what I get forever
until I blow away the cache. I see this question asked quite often regarding obtaining latest
integration versions, and I see that Ivy claims to support the local development workflow,
but I have yet to see a working example of either of these. Surely some of you have solved
these issues.

Is there any place anywhere in the world that has an example of development workflow configuration
and/or integration configuration? Can anybody provide an example? So far I have not found
anything, and advice on the net seems to be general suggestions like "try this, try that"
and nobody really has the configuration solution. Is Ivy capable of these behaviors or do
I need to hand roll a dependency management solution. Maven is too strict for these projects
(C++ code). Ivy is good if I can just obtain these fundamental behaviors.

Note that I have not provided any configs in this message that demonstrate the issues I am
having but definitely can upon request.

Thanks in advance,

L.K.


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message