Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 02EA8198FB for ; Wed, 20 Apr 2016 19:34:26 +0000 (UTC) Received: (qmail 14769 invoked by uid 500); 20 Apr 2016 19:34:25 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 14720 invoked by uid 500); 20 Apr 2016 19:34:25 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 14709 invoked by uid 99); 20 Apr 2016 19:34:25 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Apr 2016 19:34:25 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 75B502C1F60 for ; Wed, 20 Apr 2016 19:34:25 +0000 (UTC) Date: Wed, 20 Apr 2016 19:34:25 +0000 (UTC) From: "Appy (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HBASE-15651) Track our flaky tests and use them to improve our build environment MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-15651: ------------------------- Attachment: HBASE-15651-master-v4.patch > Track our flaky tests and use them to improve our build environment > ------------------------------------------------------------------- > > Key: HBASE-15651 > URL: https://issues.apache.org/jira/browse/HBASE-15651 > Project: HBase > Issue Type: Improvement > Reporter: Appy > Assignee: Appy > Attachments: HBASE-15651-master-v2.patch, HBASE-15651-master-v3.patch, HBASE-15651-master-v4.patch, HBASE-15651-master.patch, flakies.py > > > So i have written this simple script (attached) which looks at history of the [post commit build|https://builds.apache.org/view/All/job/HBase-Trunk_matrix] and outputs a list of flaky tests with some numbers. > Next steps: > 1. Setup a jenkins job (say *find-flaky-tests*) to run this script daily. We can either directly pull these results into other jobs using curl on this job's artifacts, or commit the list of flaky test to repo (idk if it's possible to commit something from jenkins job). > We'll collect results from both *post-commit* job (to add new flakies) and *flaky-tests* job (to delete tests which are no more flaky). > 2. Change *pre-commit* and *post-commit* jobs to ignore these tests using --exclude maven flag. Someone familiar with yetus might be able to do it easily. > 3. Setup a new job (say *flaky-tests*) to run only these flaky tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)