incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthias Friedrich <>
Subject Continuous Integration: Next Steps
Date Sun, 29 Jul 2012 08:02:29 GMT

over the last three weeks we've been working hard on making Crunch
ready for CI. It turned out to be much more work than initially
expected and I cursed myself more than once for starting this. In the
end it took four people and many patches to pull this off (thanks a
lot for your help, Josh, Rahul, and Gabriel!). In case you're
interested, these were the issues that lead us to CI readiness:

 * CRUNCH-8: Refactor Crunch's build into a multi-module Maven project
 * CRUNCH-17: Split unit tests from integration tests
 * CRUNCH-18: Improve build stability
 * CRUNCH-21: Make temporary directory configurable
 * CRUNCH-24: Make test suite suiteable for continuous integration
 * CRUNCH-26: Make Scrunch tests suiteable for continuous integration

We can now start with basic jobs to detect broken commits by running
the unit and integration test suites. I suggest starting with an
on-commit build using "mvn package" and a nightly build using "mvn
verify" (which runs the integration test suite). Running the
integration tests on-commit would be great but may be too expensive.

To get started we need access to Apache's Jenkins instance [1] which
apparently only PMC chairs can grant [2]. I'm new to Apache processes,
but perhaps a mentor can help us out here?

Next steps:

CI gives us the ability to regularly run reports to check code
quality. We already started preparations for this (Maven site setup,
checkstyle config and basic reports), but I think the best thing would
be to use Apache's Sonar instance [3]. I use Sonar a lot at work, it
bundles all kinds of code metrics in a nice interface and points out
where refactoring time is best spent. It's the most convenient way of
tracking code quality over time and catches many issues human
reviewers usually overlook.

Additional to reports, it's also quite common to deploy nightly
SNAPSHOT builds to a Maven repository and to publish up-to-date Maven
sites. I'm not sure if this is possible at Apache for legal reasons

Please let me know where you think we should go next. I know this
whole CI episode has been a nuisance, so thanks again for your help.



View raw message