mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pedro Larroy <pedro.larroy.li...@gmail.com>
Subject Re: Automated Flaky Test Detector
Date Wed, 01 Aug 2018 15:14:47 GMT
Hi Carl

This is great, I like a lot the design for the tool. I think it will
prevent addition of flaky tests once is implemented.
I don't have major concerns with the design, please keep us updated with
PRs about this tool.

Thanks!

Pedro.

On Tue, Jul 31, 2018 at 3:01 AM Carl Tsai <dauthdaert@gmail.com> wrote:

> Minor correction:
>
> I'm working with the *Apache* MXNet engine team at Amazon!
>
> Carl
>
> On Mon, Jul 30, 2018 at 5:42 PM Carl Tsai <dauthdaert@gmail.com> wrote:
>
> > Hi all,
> >
> >
> >
> > My name is Carl Tsai, I’m an intern working with the Amazon MXNet engine
> > team. For the past several weeks, I’ve been working with my mentor Hao
> Jin
> > on an automatic flaky test detection system. As you may know, there’s an
> > ongoing effort to reduce the number of flaky tests in the code base, and
> > this flaky test detector is meant to help ensure that these efforts will
> > have a long-lasting impact on the quality of our tests.
> >
> >
> >
> > This is an automated tool that will be run on PR checks to detect flaky
> > tests before we merge them. It will check PRs for new and modified test
> > cases and run them through a flakiness checker so that we can have high
> > confidence that our tests are not flaky. There’s a design overview on
> > confluence
> > <
> https://cwiki.apache.org/confluence/display/MXNET/Automated+Flaky+Test+Detector
> >—
> > I’d be happy to hear any feedback!
> >
> >
> >
> > In the meantime, one of the components, the flakiness checker, has been
> > merged under the tools/ folder and I’d encourage those of you who are
> > working on flaky tests or writing new tests to try it out. Documentation
> > can be found on the confluence page for the project as well as on the
> Reproducing
> > Test Results Page
> > <
> https://cwiki.apache.org/confluence/display/MXNET/Reproducing+test+results
> >
> > under tips and tricks, and I’m open to suggestions on how to improve it.
> >
> >
> >
> > Thanks,
> >
> > Carl
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message