reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mariia Mykhailova <>
Subject Re: Disable tests with transient failures
Date Mon, 10 Jul 2017 15:46:01 GMT

I still think that disabling transiently failing tests is a bad idea, for the same reasons
as before. If we're not dedicating time to figure out and fix the causes of transient failures
now when we say they impact us, then disabling the problematic tests is almost a guarantee
that we'll never look into fixing them again.

I also don't think the impact on productivity is as significant. Yes, the author of the pull
request has to look into AppVeyor test failures to see whether they are transient or caused
by his change - but this is supposed to be done anyways! CI tools are there for people to
look into them and notice when there are tests failing or RAT check not passing or anything
else is wrong, not just to provide us with the green ticks - and I don't think we do this
as systematically as we should. When I have time to review a pull request, I'm not deterred
from doing it by a failing AppVeyor, I just spend a couple of minutes to check the test failures.


From: Douglas Service <>
Sent: Sunday, July 9, 2017 3:15:33 PM
Subject: Re: Disable tests with transient failures

I agree this is very important and it is also impacting productivity. Pull
requests with transient failures appear to not be ready when in fact
they are ready for submission and would appear as such if all tests with
transient failures were disabled.


On Fri, Jul 7, 2017 at 8:00 PM, Byung-Gon Chun <> wrote:

> I agree that we have to deal with .NET test(s) that mostly fail in
> AppVeyor. This affects developers' experience. The latest PR failed on a
> .NET test.
> The Java side test failures were all fixed a while ago.
> It looks like we're almost there to fix .NET test failure cases thanks to
> Julia and the REEF committers/contributors who looked at the problems.
> Could someone share updates on .NET test failures in AppVeyor?
> This issue has been blocking a new release.
> Thanks!
> -Gon
> On Sat, Jul 8, 2017 at 9:26 AM, Douglas Service <> wrote:
> > Can we disable the tests with frequent transient failures? They are
> making
> > it difficult to get clean builds on pull requests.
> >
> > Doug
> >
> --
> Byung-Gon Chun

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message