lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Male <gento...@gmail.com>
Subject Re: being a good citizen is hard when you can't successfully run tests....
Date Mon, 17 Sep 2012 12:58:29 GMT
On Tue, Sep 18, 2012 at 12:45 AM, Dawid Weiss <dawid.weiss@gmail.com> wrote:

> I think we can even integrate hossman's suggestion and generate a
> stability report like weekly or something.
>
> I will take a look at this this week but it is definitely something that
> will require everyone's consensus.
>
What would they add in addition to the test histories you can see on
jenkins?


> Dawid
>
> Sent from mobile phone.
> On Sep 17, 2012 2:42 PM, "Michael McCandless" <lucene@mikemccandless.com>
> wrote:
>
>> I agree that a test that frequently fails, and does not get fixed, is
>> nearly pointless: everybody ignores it so it's as if the test didn't
>> exist.  And so it should be disabled.
>>
>> I say *nearly* because the failures are in fact useful to devs who do
>> have the itch/time to debug/fix them.
>>
>> So I think we need some middle ground here, where the tests keep
>> failing but only those that are interested in the failures see the
>> notifications.  We need to switch from a "push" model (any failure is
>> broadcast to everybody) to a "pull" model (those devs that want to
>> debug the failures go and check the logs), for such tests.
>>
>> When someone wants to make sure their change didn't break something
>> (Erick's original use case) then these tests should not run.
>>
>> I like Dawid's idea (a separate test plan that Jenkins runs with these
>> "difficult" tests, and it wouldn't email dev on failure).
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>> On Mon, Sep 17, 2012 at 7:58 AM, Robert Muir <rcmuir@gmail.com> wrote:
>> > On Sun, Sep 16, 2012 at 11:10 PM, Mark Miller <markrmiller@gmail.com>
>> wrote:
>> >> I get value from this test - if it was disabled, I'd probably
>> re-enable it.
>> >> would be great if it didn't fail so much, but the type of fail tells me
>> >> something.
>> >
>> > That means the assert in question isnt important at all. I'll remove it.
>> >
>> > Again my problem is the idea that having a failing build is "ok"
>> > because certain types of failures "don't matter". If they dont matter
>> > they should be removed.
>> >
>> > It causes a ton of noise when people are lazy about tests in this way,
>> > and it wastes a ton of peoples time. R
>> >
>> > Remember every time one of these tests fails it sends an email, that I
>> > must read (we don't yet have a way to put in the subject header its a
>> > SOLR test fail versus a LUCENE one, or i'd filter the solr ones and
>> > not be complaining as much).
>> >
>> > --
>> > lucidworks.com
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: dev-help@lucene.apache.org
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>


-- 
Chris Male | Open Source Search Developer | elasticsearch |
www.e<http://www.dutchworks.nl>
lasticsearch.com

Mime
View raw message