ant-ivy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Oulds <jonat...@sbdev.net>
Subject Re: buildnumber Ant task & automated updates of ivy.xml
Date Fri, 21 Nov 2008 14:22:49 GMT
You've covered quite a few points in this e-mail, let me try to 
summarise and then deal with each one in turn.  First of all let me say 
thank you for taking the time to write such an informative mail.

1- Tag profusion.  The polluting of the SVN repository with tagged 
commits from successful CI builds.  Your suggestion is not to tag, but 
to instead use the SVN version number as the build number in the ivy file.

To be honest I was worried about our build 'n' tag approach, but the 
newness of our system meant that we have never experienced the 
"unwieldy" tag folders that you mentioned.   So taking your advice I 
think moving to a system of linking against the SVN repository version 
is the best way forward.

2- Linking source (SVN) and artifact (Ivy) versions for reproducible 
builds.  This you achieve by storing a resolved ivy file and the 
corresponding SVN version number (held in a separate file) in another 
location possibly a separate SVN repository.

Whilst I like the idea of storing the SVN version number with the ivy 
file, I don't see how this could work if the resolved ivy file and SVN 
source version file are themselves versioned.  You would still require a 
version number to extract the correct version of the SVN source version 
file.

3- Subsequent issues with dependency management.  You suggest 
hard-coding depandancies into the ivy file to link to specific versions 
of each module.

I must admit I am sceptical about hard-coding dependency versions into 
ivy files, firstly I don't trust other developers to enter the correct 
version (I of course would always enter the correct version).  I also 
believe that most integration builds will be the result of bug fixes 
etc. or minor implementation changes i.e. non-breaking changes.  Any 
change to a module that breaks its API should also include a change to 
its major or minor version, not just the build number.

 From reading your post and that of Shawn Castrianni I've come to a few 
conclusions which I can hopefully implement and should enhance our 
experience with ivy.

1- No more tagging of CI builds, instead link to the SVN version via the 
  <Repository> element of the <Info> tag.

2- Let the CI auto increment the build number, there is no need for this 
to be the SVN version, so a simple increment should suffice.

3- A module should have a dynamic version number specified by a 
properties file that gives the major and minor version, this file is 
stored under SVN.  The build number is generated by CI using the ant 
buildnumber task.  This gives developers control over the major and 
minor version number when they make a breaking change to a modules API, 
but ensures integration builds get a distinct version number.

4- Dependencies should either be constrained by "latest.integration" or 
if a CI build fails due to an incompatible API then by [major.minor, 
major2.minor2]

As usual feedback is most welcome.


Mitch Gitman wrote:
> *Warning: really long post.*
> 
> Jonathan, I can give some insight based on my experience.
> 
> First off, just for perspective, the top-level decision at play is how to
> deal with integration versions, as covered in Best Practices. The two
> choices are:
> 
>    - "use a naming convention like a special suffix," i.e. do something like
>    Maven snapshots
>    - "create automatically a new version for each"
> 
> The Best Practices page recommends the second, more "promiscuous" approach.
> I would beg to differ, for reasons that go beyond what we're discussing on
> this thread, although I acknowledge it's a valid, legitimate approach. And
> well, since this is the tack you've chosen, let's go with it.
> 
> Actually, I previously worked on a team that took this second approach and
> published a new Ivy version on every successful CI build. So what we did
> then was tag promiscuously in Subversion. Suppose, for example, that our CI
> system built version 1.1.252 of a customerportal project. It would then do
> an svn/copy to create the following:
> * customerportal
>   * tags
>     * customerportal-1.1.252
> 
> For discussion's sake, let's assume we're talking about Subversion. Now, I
> know there's no one right way to use Subversion, even if you're following
> Subversion's layout conventions. The conventional way--if not the right
> way--to use the tags directory is to have it there to capture
> milestone/release builds. In the online Subversion book, "Chapter 4.
> Branching and Merging," under the "Common Branching Patterns" section, is a
> discussion of "Release Branches:"
> http://svnbook.red-bean.com/en/1.1/ch04s04.html#svn-ch-4-sect-4.4.1
> 
> Relevant passage: "The branch is tagged and released.  When testing is
> complete, /branches/1.0 is copied to /tags/1.0.0 as a reference snapshot.
> The tag is packaged and released to customers."
> 
> Now, I realize that *svn/copy* does a kind of virtual diff copy, so it's not
> like the profusion of tags with promiscuous tagging is gobbling up disk
> space. However, I can attest that the tags directory becomes quite unwieldy
> to negotiate. Also, unless you create a separate directory for traditional
> release tags, your release tags end up getting commingled with all the "CI
> successful build and Ivy publish" tags. All this strikes me as overkill and
> excessive complexity.
> 
> So within the context of "Ivy-version promiscuously," there is this
> second-level decision:
> 
>    - Tag promiscuously in version control.
>    - Do something else.
> 
> The question is, what's that something else?
> 
> One possible alternative to tagging promiscuously, while still
> Ivy-versioning promiscuously, is to rely on the global repository version
> Subversion produces every time there's a commit. The CI build could invoke
> the *svnversion *command, although there's no telling if the result is the
> actual version for that particular project's commit, and not a subsequent
> version for another project's commit that happened right after. (I'm not
> sure how the global repository version could be captured on the commit in a
> way that could be picked up by the CI server.) Anyway, the Subversion
> version could be published in its own properties file as an extra artifact
> to the Ivy repository.
> 
> Now, you're still not out of the woods yet.
> 
> Suppose customerportal 1.1 depends on customerapplication 1.0.+, which in
> turn depends on customerdomain 2.3.+.
> 
> Here's what we would have done on my previous team. In the ivy.xml file that
> gets committed to customerportal/tags/customerportal-1.1.252, we would
> hardcode a dependency for the particular build of customerapplication, say
> 1.0.92. Then if you go and look in
> customerapplication/tags/customerapplication-1.0.92, there would be a
> dependency on, say, customerdomain 2.3.147. I forget now if we actually
> captured these hardcoded dependencies in the Ivy repository, or if we just
> left them like 1.0.+ and 2.3.+. I believe we did capture this. Really, we
> would have had to.
> 
> Now, if you don't tag promiscuously, and instead rely on the global
> Subversion repository version, the question is, how do you reproduce the
> in-house (as opposed to third-party) dependencies of customerportal 1.1.252?
> How do you achieve reproducible builds?
> 
> Well, customerportal-1.1.252 in the Ivy repository could have its properties
> file with its global Subversion version. Call it svnversion.properties.
> Inside it's just one line:
> svnversion=8642
> 
> And customerportal 1.1.252's Ivy file could show a hardcoded dependency on
> customerapplication 1.0.92. Then if you go into customerapplication-1.0.92
> on the Ivy repository, you would see that it has a properties file with its
> own global Subversion version. And its Ivy file has a hardcoded dependency
> on customerdomain 2.3.147. And then under the customerdomain-2.3.147, you'd
> see something analogous. Recursively.
> 
> Now, this in itself is not giving you reproducible builds. What I'm
> describing is navigability, not reproducibility. I mean, what happens if the
> Ivy repository gets blown away? On first glance, you could keep the whole
> Ivy repository in source control. I hope we can all agree that, if you're
> doing a new Ivy version on every successful CI build, storing published
> in-house artifacts in source control is a really, really bad practice. I
> mean, that will become untenable in no time.
> 
> Here's what I believe the answer is to this problem (though I'd love to hear
> other answers): you store two things things from the Ivy repository in
> version control (preferably in its own distinct Subversion repository,
> separate from the source repository):
> 
>    - The published ivy.xml file with the hardcoded dependencies.
>    - The properties file with the global Subversion version.
> 
> At this point, you have all the information you need to re-create the entire
> Ivy repository, and to reproduce any given build. Granted, the deeper you
> have to go in your in-house dependency hierarchy, the more work you have.
> I'm sure some of this could be automated.
> 
> (Yes, I am familiar with IvySvn <http://code.google.com/p/ivysvn/>. Whether
> your build system is accessing Subversion directly as an Ivy repository is
> kind of a side question.)
> 
> You see, there are all sorts of version-control implications to this
> "version everything" approach, and I feel like they were glossed when that
> approach was recommended. Just speaking from my own experience here.
> 
> On Thu, Nov 20, 2008 at 6:00 AM, Jonathan Oulds <jonathan@sbdev.net> wrote:
> 
>> At present we tag our sources.
>>
>> But we are at an early stage of moving over to Ivy so I'd be interested to
>> hear the experiences of others.  Have others found this approach a
>> particularly bad / good idea, do others do anything different if so why?
>>
>>
>> Niklas Matthies wrote:
>>
>>> On Thu 2008-11-20 at 11:10h, Jonathan Oulds wrote on ivy-user:
>>> :
>>>
>>>> As an example we have our project source under SVN.  Our CI inspects the
>>>> project then builds and publishes if it detects any changes.  however if
we
>>>> hard coded the revision number into the ivy file then our CI machine would
>>>> have to update the ivy file with the new version and do a commit.
>>>>
>>>> That didn't sound like a good idea to us, so we headed down the route of
>>>> not having the version numbers specified in the ivy file and only resolving
>>>> the correct version number upon delivery.  Our CI machine will auto
>>>> increment the build number, when a module changes significantly we do a
>>>> special build of the module updating the major or minor version.
>>>>
>>> I'm curious: Given a build number, how do you later find the
>>> corresponding sources in SVN? Does the CI system tag the sources
>>> with the auto-incremented build number?
>>>
>>> -- Niklas Matthies
>>>
>>>
>>>
> 


Mime
View raw message