ant-ivy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oulds, Jonathan" <jonathan.ou...@intel.com>
Subject RE: Performing a release of an IVY module
Date Tue, 24 Jan 2017 17:26:37 GMT
Correct,

Your SCM contains an Ivy file which lists the dependencies that you want.  Artifactory has
an Ivy file that lists the dependencies that you actually got.

For example:
The Ivy file in your SCM may have the following dependency.
<dependency org="google" name="protobuf" rev="2.5.+"/>

However, the Ivy file in Artifactory will have the corresponding dependency.
<dependency org="google" name="protobuf" rev="2.5.0.123" revConstraint="2.5.+"/>

Always used the archived Ivy file from Artifactory for reproducible builds.



Jonathan Oulds
Snr. Software Engineer
Solution PSC Lead (PerDP)
Next Gen Data Protection


-----Original Message-----
From: Michael.Fitzner@bmw.de [mailto:Michael.Fitzner@bmw.de] 
Sent: Tuesday, January 24, 2017 4:35 PM
To: ivy-user@ant.apache.org
Subject: AW: Performing a release of an IVY module

Thanks for the feedback and you experiences. As far as I understood, the recommendation is
to use both systems SCM and Artifactory/IVY to manage all information for reproducible builds.

Preparing a release goes than through the following phases:

- Checkout sources from SCM system with dynamic IVY descriptor e.g. last.integration into
a clean workspace
- Resolve dependencies
- Build software and test it
- Create tag with the new software version and build number e.g. 1.2.3-20
- Publish artifacts with further information
	- SCM revision
	- Correct state e.g. release
	- Branch e.g. master or release_1.2.x
	- New version number e.g. 1.2.3-20

With this approach not all information about the dependencies and correct state, revision,
branch are stored inside the SCM. 

Regards,
Michael

-----Urspr√ľngliche Nachricht-----
Von: Oulds, Jonathan [mailto:jonathan.oulds@intel.com]
Gesendet: Dienstag, 24. Januar 2017 13:57
An: ivy-user@ant.apache.org
Betreff: RE: Performing a release of an IVY module

We approach this problem from the other end, using Artifactory as the source of truth instead
of SVN.

Of course we use SVN to hold our source code, but we never push generated artefacts back to
SVN, instead resolved Ivy files are written to Artifactory, see the example below:
<info organisation="my-org"
    module="my-module"
    revision="4.1.0.34.0"
    status="approved"
    publication="20170123114347"
    ext:svn.url="https://my-source-code/svn/path/to/trunk "
    ext:svn.revision="12345"
    ext:qb.id="139450"/>

This Ivy file together with the associated artefacts are written to Artifactory.  The Ivy
file identifies the SVN URL and revision number of the source code, together these allow us
to pull the correct source code from SVN when rebuilding the module.  In addition the qb.id
is a reference into our continuous integration server and can be used to discover additional
build specific information.

With this setup, you rebuild a module by first locating the relevant Ivy file in Artifactory,
next you fetch the source code and any build specific configurations (gleaned from your CI
server).  Importantly, you MUST replace the original Ivy file fetched from SVN (which is likely
to contain dynamic revision constraints) with the resolved Ivy file retrieved from Artifactory
which will only contain static revision constraints.  This ensures that when you rebuild your
module you use the same revisions of dependencies that were used when the module was originally
built.

The above Ivy snippet is from an "approved" module.  In our system this identifies a module
that has been built in an approved way, passing all relevant unit tests etc.  In order to
release a module we must first promote the module to an "candidate" status.  When promoting
a module we might run additional tests on the binaries, each time these tests pass we update
the status of the Ivy file to indicate its position within the release pipeline.

We also periodically clean redundant modules from Artifactory by deleting modules with a low
status, i.e. modules that may not have passed performance or security tests.  When deleting
modules it is important not to delete modules that are active dependencies of other modules.
 You should also be careful not to delete the most recent version of a module in the event
that you are generating build numbers based upon a last build number +1 strategy.

 

Jonathan Oulds
Snr. Software Engineer
McAfee


-----Original Message-----
From: Marc De Boeck [mailto:mdeboec@gmail.com]
Sent: Monday, January 23, 2017 9:09 PM
To: ivy-user@ant.apache.org
Subject: Re: Performing a release of an IVY module

Ivy is a flexible tool and also the release process can be tuned according to your own needs.

Personally, I think that you should not try to copy the  "release process"
used in the maven world.
I believe more in a continuous build process where builds are promoted to releases when they
have been properly tested.
A binary repository manager such as Artifactory can help with this.

Here is the build and release process that I propose for Ivy:

In the SCM system, dependencies to modules which are part of your "build-project" always have
a dynamic revision (latest.integration).
Dependencies to modules which are not part of the build-project are specified with a fixed
revision.

During the continuous build process, a "resolved" module descriptor (aka
ivy-file) is published to the binary repository manager. A "resolved"
ivy-file means there are no dynamic revisions in it. The related commit in your SCM system
is tagged with the name of the build. The status of the resolved ivy-file is "integration".

When a build has to be released (according to appropriate own test and acceptance criteria),
the resolved module descriptor is retrieved from the binary repository manager, its status
is changed to "released", and the module descriptor is published to a "release" repository.
Depending on your needs, you can do the same with all its dependencies (the exact revisions
are available in the published ivy-files). The corresponding build artifact is then promoted
in the binary repository manager. If you are using Git, you can then merge the associated
tagged commit into the release branch (this should be a simple forward merge, or else something
is messed up in your branching tree).  Additionally, you can also tag your SCM-system with
the actual release name of your product. That would be the end of the release process.

* This process implies that your binary repository manager is as important for your build
and release process as the SCM system. So it should be protected appropriately. It is also
usefull to setup a batch job to scrub build artifacts which haven't been promoted within a
specific timeframe (e.g. one month).
* Because builds and tests are done prior to the release process and not afterwards, this
release process is much lighter than the maven release process.

I am also curious how other people make their releases with Ivy. So feel free to comment and
propose alternative release process.

Regards,
Marc




2017-01-23 20:02 GMT+01:00 <Michael.Fitzner@bmw.de>:

> We are searching for a best practice documentation about the release 
> process of an IVY module like described here [1]. For us it is quite 
> important to produce reproducible builds with all dependencies of the 
> release build. For that, we have to push the resolved IVY descriptor 
> in the SCM system and create a tag. Does anybody know how this could 
> be achieved with the IVY system?
>
> - Resolve dependencies
> - Commit descriptor to the SCM system
> - Checkout source from the tag
> - Do release build and test
> - Publish module artifacts
>
> Thanks
> Michael
>
> [1] Maven release process of modules
>       http://maven.apache.org/maven-release/maven-release-plugin/
> examples/prepare-release.html
>
---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for the sole use of the
intended recipient(s). Any review or distribution by others is strictly prohibited. If you
are not the intended recipient, please contact the sender and delete all copies.
---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
Mime
View raw message