cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hugo Trippaers <h...@trippaers.nl>
Subject [JENKINS] Jenkins the spammer
Date Wed, 29 Jan 2014 08:09:05 GMT
Hey guys,

Ofter the last few weeks we had a lot of problems with Jenkins actually reporting build problems
that were actually infrastructure related. I try and explain the issues and how we try to
prevent them from happening in the future.

The GIT plugin used by jenkins has a build in timeout of 10 minutes, if a git operation takes
longer than this it will report an error. This timeout is not configurable from within Jenkins
and needs to be set with a system property. I’ve made a patch on the cloud-jenkins plugin
that will allow us to set custom system properties for the jclouds slaves. This resolved most
of the intermittent build problems.

There was also the recent change to java 7 for master. This caused most of the master builds
to fail on Jenkins as all slaves were setup to explicitly use java 6 (at that was our target
platform). Now we made some changes so each build can specify which JDK version it will use
at build time. This works on the cloudstack-buildslaves but not yet on the rpm builders. Because
of this the debian and redhat packages are not built yet.

There were also some problems with the artifacts required for the awsapi. Due to a configuration
in the poms for the transient dependencies (outside our tree) several artifacts would have
be downloaded from a maven repository that no longer existed. However the site didn’t return
a 404 that maven would pickup, but a 200 with an index page so maven would think it properly
downloaded the pom/jar file. Causing grief later in the proces. We now use a custom settings.xml
for the master build that redirects this particular repository to the archiva instance that
david setup.

The positive news is that we now have upgraded the service offering for the cloudstack-buildslaves
to 4 x 2.6Ghz cores and 4Gb ram, further reducing the main master build to just under 8 minutes
(including awsapi).

It’s not completely done yet, but hopefully this will help getting Jenkins back into a stable
state where we can rely on failures being real failures in the build and not in the infrastructure.

Cheers,

Hugo




Mime
View raw message