hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <bus...@cloudera.com>
Subject Re: Better management of flakies
Date Tue, 29 Mar 2016 23:39:38 GMT
How about if we make a script for determining flakey status, based on
rate of failing in either post-commit or in the pre-existing branch
check for precommit. The script could live in dev-support and work by
effectively grabbing the test reports from those jobs.

We can then run that script in a job and save the results as a build artifact.

We can then build the changes in your original email: ignoring the
list entries in our normal builds and rechecking just them in a
dedicated job.

On Tue, Mar 29, 2016 at 6:30 PM, Sean Busbey <busbey@cloudera.com> wrote:
> Where/How do we maintain the list of tests that are flagged as flakes?
> On Tue, Mar 29, 2016 at 6:25 PM, Apekshit Sharma <appy@cloudera.com> wrote:
>> Proposal:
>> Maintain a list of flaky tests which can be ignored in main builds (for
>> patches) and setup a new job which will periodically run those flaky tests.
>> Benefits:
>> - Cleaner main builds. Less runs per patch,  ideally no re-runs because of
>> timeouts and other flakiness. Less frustration. Increased developer
>> productivity.
>> - Logs from various runs available when anyone tries to fix these tests.
>> Possible con: We start demoting tests to flaky list which never get fixed.
>> How?
>> Internally, we have setup up two jenkins jobs, one for main build and one
>> for flaky. In execute shell step, they curl to get the list of flaky tests
>> and use surefire plugin flags to ignore/run particular tests.
>> I volunteer to set them up. However, not sure if this approach will work
>> since we have yetus upstream.
>> - Appy
> --
> busbey


View raw message