Mailing-List: contact dev-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@zookeeper.apache.org
From: Jordan Zimmerman <jordan@jordanzimmerman.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Subject: Re: Introduce ZooKeeper flaky test dashboard
Date: Tue, 11 Apr 2017 21:48:10 -0500
References: <CA+i0x1+afAe5vYrfoenxSJ15b3N=wjNTmLbCJEGfPXH=yjiDug@mail.gmail.com>
To: dev@zookeeper.apache.org
In-Reply-To: <CA+i0x1+afAe5vYrfoenxSJ15b3N=wjNTmLbCJEGfPXH=yjiDug@mail.gmail.com>
Message-Id: <AB1EC52E-AD9E-4835-BFA2-05CB3185E962@jordanzimmerman.com>
archived-at: Wed, 12 Apr 2017 02:48:25 -0000

Nice work - we could really use this on Curator. I'll be stealing this =
soon ;D

> On Apr 11, 2017, at 9:33 PM, Michael Han <hanm@cloudera.com> wrote:
>=20
> tl;dr
> =
https://builds.apache.org/job/ZooKeeper-Find-Flaky-Tests/lastSuccessfulBui=
ld/artifact/report.html
>=20
>=20
> We all know quality is important to this project. One of major =
advantage of
> ZooKeeper comparing to other similar solutions is its quality: solid,
> stable, and well tested. Unit tests are playing an important role for =
ZK's
> quality so we need take these tests seriously. In particular for =
failed
> tests, it's easy to blame they are failed because of flaky but usually
> there are bugs (sometimes very subtle ones) behind the failures.
>=20
> Inspired by some good work done on HBase project, I've borrowed and =
tweaked
> their script so it applies to ZooKeeper project. The dashboard is now =
up
> and a Jenkins job is running on a daily base to monitor our builds and =
pick
> up flaky tests. I hope this dashboard can help increase the visibility =
and
> raise awareness of the quality of unit tests, and also help developers
> track and diagnose failures.
>=20
> Any feedback will be appreciated, and of course, contributions are =
very
> welcome.
>=20
> --=20
> Cheers
> Michael.