Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: <CAFb6HiL5LhGY19f5dGy-R6E7S2gmhbhzTQ7nPztnz9B-QQFLyQ@mail.gmail.com>
References: <CAAjhxrqa3BcOaME3bTmgnyHOw1tDSAGzH=rV7JNyc_1dZDH_Kg@mail.gmail.com>
 <CADcMMgF830o7S5U0zvv6CL2yn=3FJRPdyhBiMh+b7V4SjJAecA@mail.gmail.com>
 <CADcMMgHSR8_-hOf5_20mTF92H73zAMy9X6Dfpr7Bu2DY3DwgEQ@mail.gmail.com>
 <CANZa=Gu2bODgDCeCgVMf4gGO_f19ecL=5e+5-1YOPN50ADJXfw@mail.gmail.com> <CAFb6HiL5LhGY19f5dGy-R6E7S2gmhbhzTQ7nPztnz9B-QQFLyQ@mail.gmail.com>
From: Todd Lipcon <todd@cloudera.com>
Date: Fri, 20 May 2016 15:43:58 -0700
Message-ID: <CADY20s4iS=aaG=0U3UW_HtysnbQs-VR39Svvfywnv-NJ7Ov48A@mail.gmail.com>
Subject: Re: Smart Flaky Handler
To: dev <dev@hbase.apache.org>
Content-Type: multipart/alternative; boundary=e89a8f647427c364fe05334dd4cf
archived-at: Fri, 20 May 2016 22:44:24 -0000

--e89a8f647427c364fe05334dd4cf
Content-Type: text/plain; charset=UTF-8

On Fri, May 20, 2016 at 1:17 PM, Matteo Bertozzi <theo.bertozzi@gmail.com>
wrote:

> any suggestion on how to make people aware of the tests being flaky?
>

You guys might consider doing something like what we do for Apache Kudu
(incubating):

http://dist-test.cloudera.org:8080/ has a dashboard (driven from our
flaky-tests job) which shows the percent flakiness of each test, as well as
a breakdown of pass/fail rates by revision. We don't automatically email
these to the list or anything, currently, but would be pretty easy to set
up a cron job to do so.

The dashboard is very helpful for prioritizing the de-flaking of the worst
offenders, and also useful to quickly drill down and grab failure logs from
the flaky tests themselves.

-Todd
-- 
Todd Lipcon
Software Engineer, Cloudera

--e89a8f647427c364fe05334dd4cf--