From dev-return-313144-archive-asf-public=cust-asf.ponee.io@lucene.apache.org Tue Feb 27 19:24:05 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 79A7D180651 for ; Tue, 27 Feb 2018 19:24:04 +0100 (CET) Received: (qmail 11332 invoked by uid 500); 27 Feb 2018 18:24:03 -0000 Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list dev@lucene.apache.org Received: (qmail 11321 invoked by uid 99); 27 Feb 2018 18:24:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Feb 2018 18:24:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 7DC35C19D4 for ; Tue, 27 Feb 2018 18:24:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -110.311 X-Spam-Level: X-Spam-Status: No, score=-110.311 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id WLLPhuFR2xtg for ; Tue, 27 Feb 2018 18:24:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 43BD35F254 for ; Tue, 27 Feb 2018 18:24:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 78F9BE00B5 for ; Tue, 27 Feb 2018 18:24:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 35D3724094 for ; Tue, 27 Feb 2018 18:24:00 +0000 (UTC) Date: Tue, 27 Feb 2018 18:24:00 +0000 (UTC) From: "Steve Rowe (JIRA)" To: dev@lucene.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (SOLR-12028) BadApple and AwaitsFix annotations usage MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/SOLR-12028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16379084#comment-16379084 ] Steve Rowe commented on SOLR-12028: ----------------------------------- bq. BadApple tests should be run often enough to give various summaries something to report on I think I saw weekly frequency suggested elsewhere for badapple tests, but then Hoss's 7-day summary won't have much to work with. I think daily is a better frequency. I'll set up daily job with badapple=true for master and branch_7x, and another set of weekly jobs with nightly=true & badapples=true (there are a few tests with this annotation combination), on both Apache and my Jenkins. [~thetaphi], do you want to host badapple tests on Policeman Jenkins? > BadApple and AwaitsFix annotations usage > ---------------------------------------- > > Key: SOLR-12028 > URL: https://issues.apache.org/jira/browse/SOLR-12028 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) > Components: Tests > Reporter: Erick Erickson > Assignee: Erick Erickson > Priority: Major > Attachments: SOLR-12016-buildsystem.patch, SOLR-12028.patch, SOLR-12028.patch > > > There's a long discussion of this topic at SOLR-12016. Here's a summary: > - BadApple annotations are used for tests that intermittently fail, say < 30% of the time. Tests that fail more often shold be moved to AwaitsFix. This is, of course, a judgement call > - AwaitsFix annotations are used for tests that, for some reason, the problem can't be fixed immediately. Likely reasons are third-party dependencies, extreme difficulty tracking down, dependency on another JIRA etc. > Jenkins jobs will typically run with BadApple disabled to cut down on noise. Periodically Jenkins jobs will be run with BadApples enabled so BadApple tests won't be lost and reports can be generated. Tests that run with BadApples disabled that fail require _immediate_ attention. > The default for developers is that BadApple is enabled. > If you are working on one of these tests and cannot get the test to fail locally, it is perfectly acceptable to comment the annotation out. You should let the dev list know that this is deliberate. > This JIRA is a placeholder for BadApple tests to point to between the times they're identified as BadApple and they're either fixed or changed to AwaitsFix or assigned their own JIRA. > I've assigned this to myself to track so I don't lose track of it. No one person will fix all of these issues, this will be an ongoing technical debt cleanup effort. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org