lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erick Erickson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-12028) BadApple and AwaitsFix annotations usage
Date Tue, 27 Feb 2018 18:08:00 GMT

    [ https://issues.apache.org/jira/browse/SOLR-12028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16379055#comment-16379055
] 

Erick Erickson commented on SOLR-12028:
---------------------------------------

[~steve_rowe] Fair enough. I expect it to take a bit to make this smooth. Here's the goal
as I see it, subject to negotiation. How we work it out is TBD:

Tests that are always broken are AwaitsFix. These won't be run regularly. I'm going to provide
a report each week (just an e-mail to the dev list) with the current tests with either AwaitsFix
or BadApple annotations.

BadApple tests should be run often enough to give various summaries something to report on
(e.g. Hoss' report, your work). Personally I'm going to filter them to a special folder that
I can examine at leisure.

Anything run with tests.badapples=false that fails requires immediate attention. For a short
while that may just be adding BadApple annotations. When things settle down, adding more BadApple
annotations should only rarely be done.




> BadApple and AwaitsFix annotations usage
> ----------------------------------------
>
>                 Key: SOLR-12028
>                 URL: https://issues.apache.org/jira/browse/SOLR-12028
>             Project: Solr
>          Issue Type: Task
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: Tests
>            Reporter: Erick Erickson
>            Assignee: Erick Erickson
>            Priority: Major
>         Attachments: SOLR-12016-buildsystem.patch, SOLR-12028.patch, SOLR-12028.patch
>
>
> There's a long discussion of this topic at SOLR-12016. Here's a summary:
> - BadApple annotations are used for tests that intermittently fail, say < 30% of the
time. Tests that fail more often shold be moved to AwaitsFix. This is, of course, a judgement
call
> - AwaitsFix annotations are used for tests that, for some reason, the problem can't be
fixed immediately. Likely reasons are third-party dependencies, extreme difficulty tracking
down, dependency on another JIRA etc.
> Jenkins jobs will typically run with BadApple disabled to cut down on noise. Periodically
Jenkins jobs will be run with BadApples enabled so BadApple tests won't be lost and reports
can be generated. Tests that run with BadApples disabled that fail require _immediate_ attention.
> The default for developers is that BadApple is enabled.
> If you are working on one of these tests and cannot get the test to fail locally, it
is perfectly acceptable to comment the annotation out. You should let the dev list know that
this is deliberate.
> This JIRA is a placeholder for BadApple tests to point to between the times they're identified
as BadApple and they're either fixed or changed to AwaitsFix or assigned their own JIRA.
> I've assigned this to myself to track so I don't lose track of it. No one person will
fix all of these issues, this will be an ongoing technical debt cleanup effort.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message