Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Received-SPF: pass (nike.apache.org: domain of trippie@gmail.com designates
 74.125.83.44 as permitted sender)
Sender: Trippie <trippie@gmail.com>
From: Hugo Trippaers <hugo@trippaers.nl>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable
Subject: [JENKINS] Jenkins the spammer
Message-Id: <17FD6459-BB71-4FA9-B4A0-4A45FD5E3DF5@GMAIL.com>
Date: Wed, 29 Jan 2014 09:09:05 +0100
To: dev <dev@cloudstack.apache.org>
Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\))

Hey guys,

Ofter the last few weeks we had a lot of problems with Jenkins actually =
reporting build problems that were actually infrastructure related. I =
try and explain the issues and how we try to prevent them from happening =
in the future.

The GIT plugin used by jenkins has a build in timeout of 10 minutes, if =
a git operation takes longer than this it will report an error. This =
timeout is not configurable from within Jenkins and needs to be set with =
a system property. I=92ve made a patch on the cloud-jenkins plugin that =
will allow us to set custom system properties for the jclouds slaves. =
This resolved most of the intermittent build problems.

There was also the recent change to java 7 for master. This caused most =
of the master builds to fail on Jenkins as all slaves were setup to =
explicitly use java 6 (at that was our target platform). Now we made =
some changes so each build can specify which JDK version it will use at =
build time. This works on the cloudstack-buildslaves but not yet on the =
rpm builders. Because of this the debian and redhat packages are not =
built yet.

There were also some problems with the artifacts required for the =
awsapi. Due to a configuration in the poms for the transient =
dependencies (outside our tree) several artifacts would have be =
downloaded from a maven repository that no longer existed. However the =
site didn=92t return a 404 that maven would pickup, but a 200 with an =
index page so maven would think it properly downloaded the pom/jar file. =
Causing grief later in the proces. We now use a custom settings.xml for =
the master build that redirects this particular repository to the =
archiva instance that david setup.

The positive news is that we now have upgraded the service offering for =
the cloudstack-buildslaves to 4 x 2.6Ghz cores and 4Gb ram, further =
reducing the main master build to just under 8 minutes (including =
awsapi).

It=92s not completely done yet, but hopefully this will help getting =
Jenkins back into a stable state where we can rely on failures being =
real failures in the build and not in the infrastructure.

Cheers,

Hugo