ant-ivy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From wellgoonthen <richcol...@googlemail.com>
Subject Re: External vs. Internal dependencies
Date Fri, 07 Sep 2012 12:25:26 GMT

Hi Mitch,

Thanks for your response and apologies if my terminology was not accurate. 
Based on your comments I'll try to explain the issue again.  When I talk
about A, B and C, these are all modules of the same project with a root
level build.xml calling module level builds + ivy.xml, where A depends on B
and C can depend on A, B or in fact both.  A, B and C are what I mean by
internal in the sense I own the code etc.

If I look at the ivy file of B, I may see (pseudo code for sake of the
email):

 depends on junit
 depends on log4j
 depends on A

Based on the simple uptodate logic that can be used from ant I can easily
remove the need to call ivy:resolve for junit and log4j by checking the
ivy.xml file for module B.  If it's not newer, I know that someone hasn't
changed the version of junit etc.

As the build process would be run from the top level, as B runs there very
well may have been a rebuild of the A artifact that was pushed to the local
cache on the dev's machine.  So this is where ivy:resolve is required, it
will check if this is the case.

The reason I'm trying to avoid or at least cut down the situation where I
run an ivy:resolve is due to the amount of time it takes.  If I have a
module with 35 dependencies, it can take over 1 minute in some cases where
the full module build and test time may only be 2 mins.

I hope that's a bit clearer and the purpose of this is really to focus on
local dev builds speed those up.  I'm not really targeting this from a CI
build perspective, although I imagine that if I fix the local issue, the CI
build will also benefit from the same kind of changes.

Richard






Mitch Gitman wrote:
> 
> Your question is a bit ambiguous, so let me not so much answer it as make
> two points to get the terminology and concepts straight.
> 
> When you talk about skipping the resolution of the external artifacts,
> what
> you really should be saying is that you're still doing an ivy:resolve,
> just
> you're trusting the modules in the Ivy cache are up-to-date and Ivy
> doesn't
> have to check the remote repository again. This is the default behavior,
> and the natural behavior for external libraries and released versions of
> internal libraries.
> 
> For internal libraries still under development, you want to override this
> so that Ivy always checks the repo. Typically, this is done on the
> resolver
> in ivysettings.xml, and I can think of three criteria you can pass to the
> resolver to say that it's an "integration" version:
> 
>    - The publication status in the ivy.xml is "integration" or something
>    comparable.
>    - There's a special suffix in the version, like -SNAPSHOT or a
>    buildnumber or timestamp.
>    - The modules are in a special integration repository directory.
> 
> It's not a bad idea to do all three in combination, although any one will
> do. See:
> http://ant.apache.org/ivy/history/latest-milestone/concept.html#change
> 
> When you talk about using Ivy resolve to tell if A has been rebuilt
> triggering a new build of B, you're applying concepts that apply not to a
> dependency manager like Ivy but rather to a continuous integration server
> like Jenkins. Forgive me if I'm stating something you're already well
> aware
> of, but the way that a change in A triggers a build of B is to have the CI
> server monitor for check-ins to the different projects. If someone commits
> a change to A, it triggers a build of A, and if that build is succeeding,
> a
> new version of the A artifact gets published to the repository. From here,
> the CI server should be configured (whether manually or through an Ivy
> integration) to have a successful build of A trigger a build of B. From
> here, B is assured to get the latest A that was just published.
> 
> Your question implies CI builds as opposed to developer builds, since the
> developer knows what projects they've changed or have incoming changes
> from
> source control.
> 
> On Thu, Sep 6, 2012 at 4:12 AM, wellgoonthen
> <richcoleuk@googlemail.com>wrote:
> 
>>
>> Hi,
>>
>> I'm currently doing some build doctor work on our system build.  I've
>> done
>> quite a lot of work to implement a java incremental set up using uptodate
>> etc.  This is great and saves a lot of time/work.  The biggest issue is
>> now
>> the ivy resolution.  To summarise we have two types dependencies that I
>> call:
>>
>> - External - junit, log4j etc.
>> - Internal - intra build module artifacts
>>
>> Now I can easily skip the resolution of the external artifacts by
>> checking
>> the "uptodateness" of the modules ivy.xml file to see if the version of
>> junit has been changed, or a new 3rd party lib added to that module.  The
>> issue I have is with the internal dependencies.
>>
>> For example if my build consists of modules A -> B -> C, built in that
>> order.  I currently have to rely on ivy resolve to know if A's artifact
>> has
>> been rebuilt which would therefore trigger a rebuild of B.
>>
>> To summarise I'm using ivy:resolve to do this and for one module that
>> takes
>> a total of 11 seconds, 8 seconds is spent resolving stuff.
>>
>> Does anyone have any ideas on this?
>>
>> Many thanks
>>
>> Richard Cole
>> --
>> View this message in context:
>> http://old.nabble.com/External-vs.-Internal-dependencies-tp34396743p34396743.html
>> Sent from the ivy-user mailing list archive at Nabble.com.
>>
>>
> 
> 

-- 
View this message in context: http://old.nabble.com/External-vs.-Internal-dependencies-tp34396743p34402432.html
Sent from the ivy-user mailing list archive at Nabble.com.


Mime
View raw message