geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Dillon" <ja...@planet57.com>
Subject Re: Geronimo build automation status (longish)
Date Thu, 28 Dec 2006 05:11:58 GMT
A little holiday update on build automation muck...

Turns out jolly ole st. nick brought me some coal for all of this
anthill effort.  I had thought that it would be easier/faster to get
automated TCK builds up and running with direct effect than it would
be to put more time into the GBuild codebase.

Well, months have gone by now and I'm still no where near to getting
anthill to behave... and I'm close to giving up on it.  Still gonna
see if the anthill folks can get some of the issues I've been
reporting to them fixed, and they have been very good about fixing
them and even adding a few features to support or TCK testsuite
automation.  But if not... well I can't handle the frustration
anymore... as this has already turned me into a ragged old man whining
and bitching at everyone.

Its still not all anthill's fault though... mvn really doesn't make
life any easier with its limited release capabilities and lack of
artifact promotion, network dependency and other problems.  Really...
making it really difficult to architect a rich and robust build
automation solution around it... impossible even.

In retrospect it would have been more worthwhile to work on
integrating gbuild into continuum and adding more workflow features.
Oh well... now I'm just so sick of all things build that I'm not
really sure what I'm gonna do.

Ho ho ho... click.

:-\

--jason


On 12/1/06, Jason Dillon <jason@planet57.com> wrote:
> Sorry, this has been long overdue.  I've been working on some
> automation systems for Geronimo builds, including the basic server
> assemblies, cts assemblies, tck testsuite execution as well as soon
> to run our own testsuite.
>
> I have used many different build automation platforms in the past...
> but IMO they all have some deficiency.  Anyways, I elected to
> implement a solution using AntHill, who's publisher, Urbancode, has a
> policy to allow free usage for open-source projects (just like
> Atlassian's JIRA & Confluence).
>
> I've set up the latest version of AntHill 3 on our gbuild hosts, and
> have been working on a configuration to allow reliable builds of
> Geronimo.  One of the nice aspects of AntHill3 is its distributed
> agent system, which allows workload to be split up over a set of
> nodes.  A downside to this is that it becomes more difficult to link
> Maven builds, as Maven uses a local repository cache for all
> integration points.  But, I have gotten around this issue by having
> AH include all of the artifacts downloaded and produced by a build
> into a clean local repo by the target project which is building.
>
> A nice side effect of this is that there is direct correlation
> between one build to another.  And aside from any mystical SNAPSHOT
> mismatches (which I hope to get fixed soon with my mvn patch http://
> jira.codehaus.org/browse/MNG-2681) it is fairly safe to say that
> artifacts generated/downloaded by one build will be used by a
> dependent build.  The down side to this is that sometimes we have to
> ship about ~512mb of dependencies for larger builds (like the cts-
> server builds for the TCK which depend on all of the outputs of the
> server builds, which is a local repo cache of ~512mb).
>
> An even nicer side effect to all of this, now that each build has a
> set of artifacts which can be retrieved by another process... we can
> then take a successful build of Geronimo and run our testsuite on
> it... either automatically or manually.  And when the testsuite gets
> bigger and bigger, we can split up each of the suites and run each
> one a different system... or even on a different operating system or
> architecture.
>
> Anyways... the options ahead of us are really interesting... and I
> believe that right now that AntHill3 is the best tool available to
> our community to build a really rich and powerful build automation
> system.
>
> I am however still working out some of the kinks...
>
> For example, to run our console-testsuite automatically on gbuild
> hosts, we need to setup a virtual X server for Firefox to connect to,
> which means we need to setup some tasks to execute Xfvb before tests
> and shut it down afterwards, as well as put firefox-bin on the path,
> etc.  Minor issues, but still work left to be done.
>
> If you'd like to take a peek, you can log into the AntHill console here:
>
>      https://gbuild.org:9443
>
> Username: guest
> Password: gbuild
>
> (NOTE: This user will not be able to see any of the CTS or TCK
> related projects due to NDA mucky muck)
>
> I hope to have this wrapped up for the main G server builds over the
> next few days, at which point I will enable the build status
> notifications to be sent to dev@  But right now since I am testing
> its probably not meaningful to send out those notifications.
>
> But, I have found several build related issues from testing this
> system, which is usually performed off of a clean svn co with a clean
> mvn repo... so I'm confident that once its going that we will catch
> more errors faster, which will hopefully reduce build related errors
> for the masses.
>
>   * * *
>
> Anyways, right now I have builds setup for:
>
>      Genesis - trunk
>      Specs - trunk
>      Geronimo Components (stage=bootstrap) - trunk & 1.2
>      OpenEJB 2 - trunk & 2.2
>      Geronimo Server (stage=assemble) - trunk & 1.2
>      Geronimo CTS 1.2
>
> As noted above, these builds use the exact outputs from one build in
> another, not using a shared local repo, so there is less chance that
> other builds will cause mvn to behave strangely (or stranger than it
> already does).
>
> I have started working on a workflow to run the server/testsuite/*
> modules on "Geronimo Server" outputs, which should be closer to being
> finished early next week.
>
> Some of these projects, those that generate Surefire reports, will
> have a "Surefire_Report" attached to the buildlife.  This is a
> consolidated report of all the tests for that project.  For example
> the last build of specs trunk, had 300 tests (all passed).
>
>
>
>
> NOTE: Screen shots attached, as currently the state of the install
> may change as I configure to validate the setup.
>
> You can also see & download any of the outputs of the build.
>
>
>
>
>   * * *
>
> Anyways, as mentioned this all needs a little bit more love to be
> more of the perfect build automation system which was what I have
> been really trying to put together for our community.
>
> Should have at the very least, nightly deploys of SNAPSHOTs hooked up
> for G projects by early next week.  Then nightly website updates, and
> then automated testsuite & tck bits will follow shortly afterwards...
> and eventually, we could also use AH to generate the RC and release
> builds, so that all builds are generated from the same environment.
> But probably sooner than that, we can promote G server builds that
> pass the TCK or our Testsuite, so that the exact binaries used to
> build the CTS server or run the testsuite can be used by others for
> more validation.
>
> I will send out email again later, with some brief wiki docs on what
> all this AH jargon is, and how to spin off builds with a few clicks.
>
> --jason
>
>
>
>
>
>
>

Mime
View raw message