we currently have insanely long tests, especially when it comes to
integration tests. This is not only due to the fact that our server is
slow ;), but mainly because we are using an old version of JUnit,
which run setup() and teardown() for _each_ test. We have around 3000
tests currently, with around 420 core-unit tests and 180 server-unit
Those last tests are really time consuming, as the server is started
and stopped for each single test, and that cost around 2 seconds for
each server start. It cost more than 15 minutes on my laptop ...
Junit 4, the latest version we are currently using, offers a new
annotation system where you can add a @BeforeClass and @AfterClass,
applied on a startup and teardown methods only once for all the tests
in a simple class. If we tune the tests correctly, we can then reduce
the number of server startup.shutdown to 100 startup instead if 600.
The gain will be quite important, as we may save 70% of the total
I have done some test with JUnit 3.8, using a very ugly hack, and the
result is that for the SearchTest, I went down from 46 seconds to 6
There are a few things to do now :
- I have not tested JUnit 4 right now, but there are everything we
need to speed up the test, as far as I can see
- if we are to switch to JUnit 4, we have to figure out which impact
it has on the existing code, on Eclipse integration, and more
important, on Maven (does Surefire support Junit 4 natively ?)
- another alternative would be to use the more evolved TestNG
framework ; the very same steps should be followed (see previous
- If we want to start the server only once, we have to be very careful
when writing a new test : it should not impact the existing data
loaded. That mean that a test should left the data in the same state
when it has been completed (successfully or not) than when it started.
There may be some other options, like scripting tests (I was initially
thinking about using JMeter, but it's far from being perfect), or
adding a very simple framework to compare expected entries with those
get from the server.
At this point, I would like to get your opinion.