lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dawid Weiss <>
Subject [important] Refactoring of tests infrastructure.
Date Fri, 17 Feb 2012 17:56:39 GMT
Hi everyone,

as part of this:

I've been working on moving away from ant-macros to a custom ANT task;
potential benefits are mentioned as part of the issue's description.

There is a lot of code I already pruned from LuceneTestCase as part of
this issue. Incidentally I also discovered a lot of places where the
random field proliferates to where it shouldn't be used. These are:

1) static initializers, including static field initializers;
2) constructors and field initializers in classes;

Anything using random should be placed either in setUp() override or a
custom @Before/ @BeforeClass method. The reason for this is deeply
embedded in how JUnit4 instantiates  classes and instances for tests;
I'd rather not go in the details, but it's really hard to make it work
right with static initializers or field initializers.

I've modified LuceneTestCase and added two methods instead of the
omnipresent 'random' field:

public static Random getStaticRandom() {...}
public          Random getRandom() {...}

these are returning static random context and instance (per-test)
random instance. They also throw exceptions if invoked out-of-scope
(for example from a static initializer). As you may imagine the patch
that adjusts to the above is huge and touches nearly every single test
class. I believe it's for the better -- randoms shouldn't be
uncontrollably shared and this is a first step to enforce that.

I have a working (after the above modifications) version of the trunk at:

and I would like to commit it in as soon as I can so that I don't have
to merge in changes from the trunk's tests all the time :) However,
this patch also modifies certain things everyone has probably gotten
used to:

a) the test seed format is (or rather will) change. It will be simpler
(single number), everything else will be derived from that single
b) the filtering patterns will change. no more -Dtestpackage,
-Dtestpackageroot, etc. The filtering is based on globs now (both at
method and at class level).
c) the output from 'ant test' is slightly different (details in issue
comments'). In my opinion it is clearer. There are also additional
files emitted that can help in diagnosing errors:
tests-report.txt (like the console), tests-failures.txt (errors and
failures only), tests-timehints.txt (hints for the balancer; can be
used to update static snapshot contained in the project). I also emit
junit-report compatible XMLs, but don't know if these are useful for
anything (doesn't seem like we generate those reports).

As for pt (b) above, I've added 'ant test-help' target because I
constantly forget the command line; you can see how filters have
changed, for example:

     [echo] #
     [echo] # Test case filtering:
     [echo] # - 'tests.filter' is a class-filtering shell-like glob pattern.
     [echo] # - 'tests.filter.method' is a method-filtering shell-like
glob pattern.
     [echo] #
     [echo] # Run a single test case (verbose)
     [echo] ant test -Dtests.filter=org.apache.lucene.package.ClassName
     [echo] # Run a single test case (simpler)
     [echo] ant test -Dtests.filter=*.ClassName
     [echo] # Run all tests in a package and sub-packages
     [echo] ant test -Dtests.filter=org.apache.lucene.package.*
     [echo] # Run all test methods that contain 'esi' substring
     [echo] ant test -Dtests.filter.method=*esi*
     [echo] #
     [echo] # Load balancing and caches.
     [echo] #
     [echo] # Run with a constant suite order on slave JVM (no job stealing).
     [echo] ant -Djunit4.dynamicAssignmentRatio=0 test
     [echo] # Update global (versioned) cache (from the top-level).
     [echo] ant clean test
     [echo] ant -f lucene/build.xml test-updatecache
     [echo] #
     [echo] # Miscellaneous
     [echo] #
     [echo] # Run all tests without stopping on errors (can be
diagnosed from logs).
     [echo] # from the top level:
     [echo] ant -Djunit4.haltonfailure=false test

Voices of constructive criticism (and support...) are very welcome. I
will keep working on this for the next few days (there are still some
cleanups to do), but I didn't want to catch you by surprise with such
a big change.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message