ant-ivy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Loren Kratzke <LKrat...@blueorigin.com>
Subject RE: Difficult with concurrent builds
Date Tue, 07 Apr 2015 16:16:46 GMT
Your comment about Ivy not always delivering the latest artifact caught my attention. That
in combination with perceived cache issues makes me just shake my head.

Deleting cache directories is only treating the symptoms. When you think about it, what is
the point of a cache if you delete it? Furthermore, you consider the cache poison to the build
because it is. It's a bad place to be. I have been there my friend.

See my recommendations on how to take absolute control of your dependency resolution and retrieval
strategies in Ivy. Use it, like it, then scream it from the mountain tops until it becomes
documentation on the site. It's good stuff.

http://mail-archives.apache.org/mod_mbox/ant-ivy-user/201504.mbox/%3C2ffef9f4335143be92cd684f2199808c%40MAIL02.blueorigin.com%3E

L.K.

-----Original Message-----
From: Qazwart [mailto:qazwart@gmail.com] 
Sent: Tuesday, April 07, 2015 8:48 AM
To: ivy-user@ant.apache.org
Subject: Re: Difficult with concurrent builds

Here's how I fixed the ivy cache issue with Jenkins:

In my ivysettings.xml:

<ivysettings>
    <property name="env.EXECUTOR_NUMBER" value="0" override="false"/>
    <caches
	defaultCacheDir="${ivy.default.ivy.user.dir}/cache-${env.EXECUTOR_NUMBER}"
	resolutionCacheDir="${ivy.dir}/../target/ivy.cache"/>
    <settings defaultResolver="default"/>
    <include file="${ivy.dir}/ivysettings-public.xml"/>
    <include url="${ivy.default.settings.dir}/ivysettings-shared.xml"/>
    <include url="${ivy.default.settings.dir}/ivysettings-local.xml"/>
    <include url="${ivy.default.settings.dir}/ivysettings-main-chain.xml"/>
    <include url="${ivy.default.settings.dir}/ivysettings-default-chain.xml"/>
</ivysettings>

Jenkins sets the "EXECUTOR_NUMBER" environment variable depending on which executor is used.
If the build is run locally, the default cache will be  $HOME/.ivy2/cache-0.

I ran into this cache issue because we clean out the cache from our locally built jars with
each build. This guarantees we're working with the latest version of those jars (something
that Ivy didn't always do all the time). Deleting jars while an Ivy build is running doesn't
make Jenkins very happy. It took us a while to realize why builds were failing.

--
David Weintraub
qazwart@gmail.com

perl -e 'print "Just another second rate Perl Hacker\n";'

> On Apr 7, 2015, at 9:58 AM, Roberto C. Sánchez <roberto@connexer.com> wrote:
> 
> On Tue, Apr 07, 2015 at 09:53:08AM +0200, Geißler, Daniel wrote:
>> Hi Roberto,
>> 
>> did you have a look into locking strategies, as far as i know ivy does no locking
by default wich may lead to your problems:
>> 
>> http://ant.apache.org/ivy/history/latest-milestone/settings/lock-stra
>> tegies.html
>> 
> Hi Daniel,
> 
> Thank you for responding.  This appears to be a low traffic list :-)
> 
> I did not realize that multiple builds sharing the cache could present 
> a problem.  In any event, each of my builds is configured to use its 
> own Ivy cache directory based on the Jenkins workspace directory.  I 
> don't think that affects the part of the build with which I am 
> currently experiencing a problem.  Additionally, the documentation 
> link you provided does not give an example of how I would use the 
> locking strategy.
> 
> I think that the problem has something to do with the deliver/publish 
> logic, but I cannot seem to figure out precisely what or where.  I 
> have not observed any problem with any of my downloaded artifacts for 
> the dependencies.  I have only seen a problem with the 
> delivered/published ivy.xml pertaining to the project being built at 
> the same time as it is being built by another process (but on a different branch).
> 
> Regards,
> 
> -Roberto
> --
> Roberto C. Sánchez
> http://people.connexer.com/~roberto
> http://www.connexer.com


Mime
View raw message