lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tommaso Teofili <>
Subject Re: [important] Refactoring of tests infrastructure.
Date Fri, 17 Feb 2012 22:16:51 GMT
Nice improvements! +1

2012/2/17 Dawid Weiss <>

> Hi everyone,
> as part of this:
> I've been working on moving away from ant-macros to a custom ANT task;
> potential benefits are mentioned as part of the issue's description.
> There is a lot of code I already pruned from LuceneTestCase as part of
> this issue. Incidentally I also discovered a lot of places where the
> random field proliferates to where it shouldn't be used. These are:
> 1) static initializers, including static field initializers;
> 2) constructors and field initializers in classes;
> Anything using random should be placed either in setUp() override or a
> custom @Before/ @BeforeClass method. The reason for this is deeply
> embedded in how JUnit4 instantiates  classes and instances for tests;
> I'd rather not go in the details, but it's really hard to make it work
> right with static initializers or field initializers.
> I've modified LuceneTestCase and added two methods instead of the
> omnipresent 'random' field:
> public static Random getStaticRandom() {...}
> public          Random getRandom() {...}
> these are returning static random context and instance (per-test)
> random instance. They also throw exceptions if invoked out-of-scope
> (for example from a static initializer). As you may imagine the patch
> that adjusts to the above is huge and touches nearly every single test
> class. I believe it's for the better -- randoms shouldn't be
> uncontrollably shared and this is a first step to enforce that.
> I have a working (after the above modifications) version of the trunk at:
> and I would like to commit it in as soon as I can so that I don't have
> to merge in changes from the trunk's tests all the time :) However,
> this patch also modifies certain things everyone has probably gotten
> used to:
> a) the test seed format is (or rather will) change. It will be simpler
> (single number), everything else will be derived from that single
> number.
> b) the filtering patterns will change. no more -Dtestpackage,
> -Dtestpackageroot, etc. The filtering is based on globs now (both at
> method and at class level).
> c) the output from 'ant test' is slightly different (details in issue
> comments'). In my opinion it is clearer. There are also additional
> files emitted that can help in diagnosing errors:
> tests-report.txt (like the console), tests-failures.txt (errors and
> failures only), tests-timehints.txt (hints for the balancer; can be
> used to update static snapshot contained in the project). I also emit
> junit-report compatible XMLs, but don't know if these are useful for
> anything (doesn't seem like we generate those reports).
> As for pt (b) above, I've added 'ant test-help' target because I
> constantly forget the command line; you can see how filters have
> changed, for example:
> test-help:
>     [echo]
>     [echo] #
>     [echo] # Test case filtering:
>     [echo] # - 'tests.filter' is a class-filtering shell-like glob pattern.
>     [echo] # - 'tests.filter.method' is a method-filtering shell-like
> glob pattern.
>     [echo] #
>     [echo]
>     [echo] # Run a single test case (verbose)
>     [echo] ant test -Dtests.filter=org.apache.lucene.package.ClassName
>     [echo]
>     [echo] # Run a single test case (simpler)
>     [echo] ant test -Dtests.filter=*.ClassName
>     [echo]
>     [echo] # Run all tests in a package and sub-packages
>     [echo] ant test -Dtests.filter=org.apache.lucene.package.*
>     [echo]
>     [echo] # Run all test methods that contain 'esi' substring
> (...r*esi*ze...).
>     [echo] ant test -Dtests.filter.method=*esi*
>     [echo]
>     [echo] #
>     [echo] # Load balancing and caches.
>     [echo] #
>     [echo]
>     [echo] # Run with a constant suite order on slave JVM (no job
> stealing).
>     [echo] ant -Djunit4.dynamicAssignmentRatio=0 test
>     [echo]
>     [echo] # Update global (versioned) cache (from the top-level).
>     [echo] ant clean test
>     [echo] ant -f lucene/build.xml test-updatecache
>     [echo]
>     [echo] #
>     [echo] # Miscellaneous
>     [echo] #
>     [echo]
>     [echo] # Run all tests without stopping on errors (can be
> diagnosed from logs).
>     [echo] # from the top level:
>     [echo] ant -Djunit4.haltonfailure=false test
>     [echo]
> Voices of constructive criticism (and support...) are very welcome. I
> will keep working on this for the next few days (there are still some
> cleanups to do), but I didn't want to catch you by surprise with such
> a big change.
> Dawid
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View raw message