zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: Introduce ZooKeeper flaky test dashboard
Date Wed, 12 Apr 2017 22:56:44 GMT
Great stuff, thanks Michael. Is there a way to indicate, for each of the
jobs, how many times the job failed vs the number of times it was run
during the time period? I often see that when one test fails it will cause
others to fail...

Patrick

On Tue, Apr 11, 2017 at 8:07 PM, Edward Ribeiro <edward.ribeiro@gmail.com>
wrote:

> Very cool feature! Congratulations for initiative! :D
>
> Em 11 de abr de 2017 11:48 PM, "Jordan Zimmerman" <
> jordan@jordanzimmerman.com> escreveu:
>
> > Nice work - we could really use this on Curator. I'll be stealing this
> > soon ;D
> >
> > > On Apr 11, 2017, at 9:33 PM, Michael Han <hanm@cloudera.com> wrote:
> > >
> > > tl;dr
> > > https://builds.apache.org/job/ZooKeeper-Find-Flaky-Tests/
> > lastSuccessfulBuild/artifact/report.html
> > >
> > >
> > > We all know quality is important to this project. One of major
> advantage
> > of
> > > ZooKeeper comparing to other similar solutions is its quality: solid,
> > > stable, and well tested. Unit tests are playing an important role for
> > ZK's
> > > quality so we need take these tests seriously. In particular for failed
> > > tests, it's easy to blame they are failed because of flaky but usually
> > > there are bugs (sometimes very subtle ones) behind the failures.
> > >
> > > Inspired by some good work done on HBase project, I've borrowed and
> > tweaked
> > > their script so it applies to ZooKeeper project. The dashboard is now
> up
> > > and a Jenkins job is running on a daily base to monitor our builds and
> > pick
> > > up flaky tests. I hope this dashboard can help increase the visibility
> > and
> > > raise awareness of the quality of unit tests, and also help developers
> > > track and diagnose failures.
> > >
> > > Any feedback will be appreciated, and of course, contributions are very
> > > welcome.
> > >
> > > --
> > > Cheers
> > > Michael.
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message