lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: being a good citizen is hard when you can't successfully run tests....
Date Mon, 17 Sep 2012 12:42:01 GMT
I agree that a test that frequently fails, and does not get fixed, is
nearly pointless: everybody ignores it so it's as if the test didn't
exist.  And so it should be disabled.

I say *nearly* because the failures are in fact useful to devs who do
have the itch/time to debug/fix them.

So I think we need some middle ground here, where the tests keep
failing but only those that are interested in the failures see the
notifications.  We need to switch from a "push" model (any failure is
broadcast to everybody) to a "pull" model (those devs that want to
debug the failures go and check the logs), for such tests.

When someone wants to make sure their change didn't break something
(Erick's original use case) then these tests should not run.

I like Dawid's idea (a separate test plan that Jenkins runs with these
"difficult" tests, and it wouldn't email dev on failure).

Mike McCandless

http://blog.mikemccandless.com

On Mon, Sep 17, 2012 at 7:58 AM, Robert Muir <rcmuir@gmail.com> wrote:
> On Sun, Sep 16, 2012 at 11:10 PM, Mark Miller <markrmiller@gmail.com> wrote:
>> I get value from this test - if it was disabled, I'd probably re-enable it.
>> would be great if it didn't fail so much, but the type of fail tells me
>> something.
>
> That means the assert in question isnt important at all. I'll remove it.
>
> Again my problem is the idea that having a failing build is "ok"
> because certain types of failures "don't matter". If they dont matter
> they should be removed.
>
> It causes a ton of noise when people are lazy about tests in this way,
> and it wastes a ton of peoples time. R
>
> Remember every time one of these tests fails it sends an email, that I
> must read (we don't yet have a way to put in the subject header its a
> SOLR test fail versus a LUCENE one, or i'd filter the solr ones and
> not be complaining as much).
>
> --
> lucidworks.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message