lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <>
Subject Re: being a good citizen is hard when you can't successfully run tests....
Date Mon, 17 Sep 2012 13:54:37 GMT
It's not so simple. If the replication test fails with the common fail I tend to see of somehow
a searcher not getting closed, I know that's not a big deal for that test. 

If it failed on something else, I know there might actually be a problem. 

It has value beyond anyone wanting to debug or fix it. 

The other ideas about how to deal with this are much better than this terrible disable idea.
That test in particular is critical coverage. The common fails I see from it (though pretty
much never locally) are not at all critical to me, and so relatively low on my already huge
priority list.

The test certainly has major value beyond people being interested in debugging. If it never
passes for you after you make changes, that's a big deal. On my machines it passes 99.9% of
the time. 

And no, I did not write the test...

Sent from my iPad

On Sep 17, 2012, at 8:42 AM, Michael McCandless <> wrote:

> I agree that a test that frequently fails, and does not get fixed, is
> nearly pointless: everybody ignores it so it's as if the test didn't
> exist.  And so it should be disabled.
> I say *nearly* because the failures are in fact useful to devs who do
> have the itch/time to debug/fix them.
> So I think we need some middle ground here, where the tests keep
> failing but only those that are interested in the failures see the
> notifications.  We need to switch from a "push" model (any failure is
> broadcast to everybody) to a "pull" model (those devs that want to
> debug the failures go and check the logs), for such tests.
> When someone wants to make sure their change didn't break something
> (Erick's original use case) then these tests should not run.
> I like Dawid's idea (a separate test plan that Jenkins runs with these
> "difficult" tests, and it wouldn't email dev on failure).
> Mike McCandless
> On Mon, Sep 17, 2012 at 7:58 AM, Robert Muir <> wrote:
>> On Sun, Sep 16, 2012 at 11:10 PM, Mark Miller <> wrote:
>>> I get value from this test - if it was disabled, I'd probably re-enable it.
>>> would be great if it didn't fail so much, but the type of fail tells me
>>> something.
>> That means the assert in question isnt important at all. I'll remove it.
>> Again my problem is the idea that having a failing build is "ok"
>> because certain types of failures "don't matter". If they dont matter
>> they should be removed.
>> It causes a ton of noise when people are lazy about tests in this way,
>> and it wastes a ton of peoples time. R
>> Remember every time one of these tests fails it sends an email, that I
>> must read (we don't yet have a way to put in the subject header its a
>> SOLR test fail versus a LUCENE one, or i'd filter the solr ones and
>> not be complaining as much).
>> --
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message