lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dawid Weiss <>
Subject Re: Annotation for "run this test, but don't fail build if it fails" ?
Date Sun, 06 May 2012 19:38:42 GMT
> If the tests are run many times a day, it would be great to get a
> daily report of the percent of time the tests pass.  Then if it goes
> from 5% to 50%, we can go uh-oh...

Yeah, well... but this is beyond the runner as it aggregates over time
-- it looks like a jenkins plugin that would analyze test run logs and
provide such statistics. I also admit I've never seen anything like
this -- a suite of tests with an allowed failure ratio over time and a
threshold that would trigger a warning...

> The crux of the problem remains that (for solr devs) it's still much
> more useful to have a test fail intermittently than to disable and not
> run the test at all.

These are weird tests if they allow for a (predictable?) failure from
time to time. I don't say it's a bad concept, but I think unit tests
may not be a good framework for handling this.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message