reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Douglas Service <dsop...@gmail.com>
Subject Re: Disable tests with transient failures
Date Mon, 10 Jul 2017 18:33:38 GMT
How does one flag a pull request in a way that committers can see all test
failures are transient and the request is ready for review?

Doug

On Mon, Jul 10, 2017 at 8:46 AM, Mariia Mykhailova <
mamykhai@microsoft.com.invalid> wrote:

> -1
>
>
> I still think that disabling transiently failing tests is a bad idea, for
> the same reasons as before. If we're not dedicating time to figure out and
> fix the causes of transient failures now when we say they impact us, then
> disabling the problematic tests is almost a guarantee that we'll never look
> into fixing them again.
>
>
> I also don't think the impact on productivity is as significant. Yes, the
> author of the pull request has to look into AppVeyor test failures to see
> whether they are transient or caused by his change - but this is supposed
> to be done anyways! CI tools are there for people to look into them and
> notice when there are tests failing or RAT check not passing or anything
> else is wrong, not just to provide us with the green ticks - and I don't
> think we do this as systematically as we should. When I have time to review
> a pull request, I'm not deterred from doing it by a failing AppVeyor, I
> just spend a couple of minutes to check the test failures.
>
>
> -Mariia
>
> ________________________________
> From: Douglas Service <dsopsrc@gmail.com>
> Sent: Sunday, July 9, 2017 3:15:33 PM
> To: dev@reef.apache.org
> Subject: Re: Disable tests with transient failures
>
> I agree this is very important and it is also impacting productivity. Pull
> requests with transient failures appear to not be ready when in fact
> they are ready for submission and would appear as such if all tests with
> transient failures were disabled.
>
> Doug
>
> On Fri, Jul 7, 2017 at 8:00 PM, Byung-Gon Chun <bgchun@gmail.com> wrote:
>
> > I agree that we have to deal with .NET test(s) that mostly fail in
> > AppVeyor. This affects developers' experience. The latest PR failed on a
> > .NET test.
> >
> > The Java side test failures were all fixed a while ago.
> > It looks like we're almost there to fix .NET test failure cases thanks to
> > Julia and the REEF committers/contributors who looked at the problems.
> > Could someone share updates on .NET test failures in AppVeyor?
> >
> > This issue has been blocking a new release.
> >
> > Thanks!
> > -Gon
> >
> > On Sat, Jul 8, 2017 at 9:26 AM, Douglas Service <dsopsrc@gmail.com>
> wrote:
> >
> > > Can we disable the tests with frequent transient failures? They are
> > making
> > > it difficult to get clean builds on pull requests.
> > >
> > > Doug
> > >
> >
> >
> >
> > --
> > Byung-Gon Chun
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message