incubator-deltacloud-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michal Fojtik <>
Subject Re: Hudson under the weather ?
Date Thu, 17 Mar 2011 16:57:29 GMT

So after 7 hours of pain suffered from Hudson configuration I finally managed
Apache Hudson to work (and I hope it will work forever;-)


 * Apache Hudson (
   - Broken for 3-4 months after upgrade on 'vesta' slave. A lot of Ruby dependencies
     was missing (like libxml2, libxslt...)

 * EC2 Hudson ( (down at this moment)
   - This one was working until 'aws' team decided to remove bundled 'activesupport'
     in this gem. After this change, our build start failing with nasty recursive loops
     caused by conflict between activesupport '.to_json' implementation and json '.to_json'

So I decide to spend some time getting this Hudson business to work again but in more 'nicer
First I removed all steps we used before to build deltacloud-core and replaced it with:

#!/bin/bash -l
rvm use 1.8.7
rvm gemset create ${BUILD_TAG}
rvm gemset use ${BUILD_TAG}
gem install bundler --no-rdoc --no-ri
cd ${WORKSPACE}/deltacloud/trunk/server
bundle config build.nokogiri --with-xml2-lib=$HOME/.rvm/usr/lib --with-xml2-include=$HOME/.rvm/usr/include/libxml2
--with-xslt-lib=$HOME/.rvm/usr/lib --with-xslt-include=$HOME/.rvm/usr/include
bundle install 
bundle check

It looks ugly but actually very powerful. To configure slave capable to build deltacloud-core,
we need to do:

 * Install Ruby 1.8.7 using RVM (rvm install 1.8.7)
 * Install libxml2 and libxslt packages (rvm package install libxml2 libxslt)

Then we have everything we need to make an isolated build. In first step I create a new gemset,
with name set to BUILD_TAG (which is Hudson variable).
Then I installed bundler to be available in this new gemset and configure bundler to
build nokogiri gem using RVM packages.
After this 'bundle install' will take care of installing all dependencies needed to run
deltacloud-core build (including gems needed for testing, reports, etc...)

#!/bin/bash -l
rvm use 1.8.7@${BUILD_TAG} 
cd ${WORKSPACE}/deltacloud/trunk/client
rake fixtures:clean 
rm -rf /var/tmp/deltacloud-mock-hudson
rake fixtures

This will clean up mock fixtures folder and generate new fixtures for testing.

#!/bin/bash -l
rvm use 1.8.7@${BUILD_TAG} 
cd ${WORKSPACE}/deltacloud/trunk/tests
export BUNDLE_GEMFILE="../server/Gemfile"
API_DRIVER="mock" bundle exec rake junit --trace

This will execute Cucumber tests using 'rake junit'. This special Rake task will produce jUnit
compatible with Hudson (so Hudson can identify failed test).

# Same as step5, just API_DRIVER is set to 'ec2'

#!/bin/bash -l
rvm use 1.8.7@${BUILD_TAG} 
cd ${WORKSPACE}/deltacloud/trunk/server
export CI_REPORTS=tmp/junit_reports/
bundle exec rake ci:setup:testunit test --trace

This will execute our Test::Unit suite and produce jUnit reports using ci_reporter gem.

#!/bin/bash -l
rvm use 1.8.7@${BUILD_TAG} 
cd ${WORKSPACE}/deltacloud/trunk/client
export BUNDLE_GEMFILE="../server/Gemfile"
bundle exec rake package --trace

This step will produce package for client (deltacloud-client-xxx.gem). Package for server
is produced
after Test::Unit.

#!/bin/bash -l
rvm use 1.8.7
rvm --force gemset delete ${BUILD_TAG} 

And finally remove gemset. This will be executed only after successful build. So in case when
something fail,
we will have exact gemset which failed (gem versions).

I hope this will work for a long time. The only thing which is not here yet is client testing.
This require to have API server running somewhere and I'm not sure if I can launch DC API
Apache slave (vesta).

I hope this will be useful for testing on different ruby platforms as well. I can install
jruby or
ruby 1.9 on RVM as well and execute tests here.

  -- Michal

On Mar 17, 2011, at 12:19 AM, David Lutterkort wrote:

> I just noticed that we haven't had a successful Hudson build in quite
> some time [1]. From the looks of it, the build is failing because of a
> missing cucumber on the build machine. 
> Any idea what went wrong there ?
> David
> [1]

Michal Fojtik,
Deltacloud API:

View raw message