hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Appy (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-15651) Track our flaky tests and use them to improve our build environment
Date Sat, 16 Apr 2016 02:16:25 GMT

     [ https://issues.apache.org/jira/browse/HBASE-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Appy updated HBASE-15651:
    Attachment: HBASE-15651-master.patch

Here's a formal patch.
I have made the following changes:
- Handle multiple-configuration jobs. Se we can directly pass 'https://builds.apache.org/view/All/job/HBase-Trunk_matrix'
- Pass multiple jobs by separating urls by space. This is needed because in final setup, we'll
need to run it on two jenkins jobs: flaky-tests job (which runs only flaky tests) and post-commit
- max runs flag : So if jenkins has history of many runs, we can tune this know to only consider
some amount of recent ones.
- '-mvn' flag: This will write two strings (to different files) which can be directly passed
to -Dtest and -Dtest.exclude.pattern maven flags to include/exclude flaky tests. These files
will be archived as artifacts and can be directly curl'ed into other jenkins jobs.

> Track our flaky tests and use them to improve our build environment
> -------------------------------------------------------------------
>                 Key: HBASE-15651
>                 URL: https://issues.apache.org/jira/browse/HBASE-15651
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Appy
>            Assignee: Appy
>         Attachments: HBASE-15651-master.patch, flakies.py
> So i have written this simple script (attached) which looks at history of the [post commit
build|https://builds.apache.org/view/All/job/HBase-Trunk_matrix] and outputs a list of flaky
tests with some numbers.
> Next steps:
> 1. Setup a jenkins job (say *find-flaky-tests*) to run this script daily. We can either
directly pull these results into other jobs using curl on this job's artifacts, or commit
the list of flaky test to repo (idk if it's possible to commit something from jenkins job).
> We'll collect results from both *post-commit* job (to add new flakies) and *flaky-tests*
job (to delete tests which are no more flaky).
> 2. Change *pre-commit* and *post-commit* jobs to ignore these tests using --exclude maven
flag. Someone familiar with yetus might be able to do it easily.
> 3. Setup a new job (say *flaky-tests*) to run only these flaky tests.

This message was sent by Atlassian JIRA

View raw message