activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Frazier <>
Subject Re: [DISCUSS] Improving ActiveMQ Tests
Date Sun, 01 Feb 2015 01:35:20 GMT
+1, this seems like a great idea Art.

I'll be glad to help with this.

> On Jan 31, 2015, at 5:25 PM, artnaseef <> wrote:
> *Background*
> As we all know, ActiveMQ tests take a very long time to execute, and a
> number of the tests have been unreliable.  Personally, it was ActiveMQ that
> lead me to learn maven's "skip test" features early on because I would not
> wait hours to get a small change into a build during development, nor did I
> want to spend time (as a user of ActiveMQ - this is before I was a
> committer) trying to resolve test failures.  This contradicts industry
> best-practices.
> A large, world-class product such as ActiveMQ relies, wisely, on automated
> tests to ensure regression bugs are not introduced with new feature
> additions and bug fixes.
> ActiveMQ unit tests really contains a variety of test types, such as:
> Use Case tests
> Load tests
> Bug Reproduction tests
> Unit Tests that target a single class (i.e. True Unit Tests)*
> * I personally haven't seen any of these in ActiveMQ, but I suspect/hope
> that some exist.
> *Goals*
> Improve the automated testing with the following result in mind:
> Developer cycle time: reduce the amount of time to execute tests when making
> changes to ActiveMQ code so that a developer gets a reasonable expectation
> that changes are valid without drawing the development cycle out into
> excessive time periods
> Continued quality: continue to cover all of the test scenarios currently
> covered
> Release validation: continue to provide very thorough validation for
> releases at the expense of significant, additional test time
> Reliability: improve reliability of tests for the release process so that
> it is reasonable to expect a good build every time for quality releases
> Verify units: improve overall quality of the system through more thorough
> testing of each unit (class) so that small bugs in units do not turn into
> hard-to-find bugs in the system (I found one of these just walking through
> code yesterday - it will get fixed soon)
> *Ideas*
> Identify junit tests that are really integration or load tests and separate
> as such  
> Create an artifact for integration tests, move integration tests there and
> run them during the verify phase  
> Create an artifact for load tests, move load tests there and also run
> during the verify phase (interim)  
> Stand up a true load-test environment and move load tests into that
> environment and out of the main build process
> Create true unit tests with high code coverage
> Move tests into separate artifacts to reduce the number of tests, and
> therefore the amount of time necessary to resume tests from a failure point
> (e.g. split activemq-unit-test into activemq-unit-test01, ...02, etc)
> Review problematic tests and determine ways to resolve (e.g. replace with
> new tests using different approaches, use the load test environment, etc)
> *Wrap-Up*
> The steps above will move ActiveMQ testing toward a solid, reliable test
> framework without creating gaps in quality.
> Unless alternative plans are defined, I am going to enter Jira tickets for
> these and start moving in this direction.  Volunteers are welcome and
> greatly appreciated.
> --
> View this message in context:
> Sent from the ActiveMQ - Dev mailing list archive at

View raw message