accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Separation of timing/performance tests from normal Maven build
Date Wed, 01 Jul 2015 18:09:53 GMT
Was talking with Eric off-list about a recent test he added.

Over the past two major release lines (1.6 and 1.7), there's been a 
significant level of effort put forth by multiple devs to get the 
integration tests running on "terrible" hardware. This has been a great 
endeavor because our tests have never been more stable and it's even 
helped us catch bugs that we would have otherwise assumed as transiently 
failing (ACCUMULO-3859 is a great example).

Because we are writing a database, we're always concerned about 
performance regressions, both high-level and low-level. I'd like to 
propose that we recognize and accept this head-on and try to move these 
specifically "high-load" and "performance related" tests to their own 
execution phase that we can run specifically on nodes that meet the 
necessary preconditions.

Some examples of tests:

DeleteTableDuringSplitIT
DurabilityIT
ManySplitIT
RollWALPerformanceIT

I know we can do some classification of tests via surefire/failsafe 
which should roughly meet our goals (typically via an annotation on the 
class). Thus, we could add a specific flag to a Maven build that would 
include this subset of tests.

What do people think?  Do others also think that this is worth pursuing?

- Josh

Mime
View raw message