ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Pavlov <dpav...@apache.org>
Subject Re: Flaky tests
Date Wed, 27 Feb 2019 12:13:17 GMT
Thanks, Ilya for your positive feedback.

Stanislav,

Apache Ignite TeamCity Bot requires CI credentials, so it may be easier to
check its code and description here
https://github.com/apache/ignite-teamcity-bot#ignite-teamcity-bot

(For Russian native speakers there is a blog post here
https://habr.com/ru/company/sberbank/blog/436070/ , it is not yet
translated to English)

As for Java 11, there is an idea to separate stable/flaky statistics from
the master for two Java versions, but it is only planned now.

Sincerely,
Dmitriy Pavlov

ср, 27 февр. 2019 г. в 12:01, Ilya Kasnacheev <ilya.kasnacheev@gmail.com>:

> Hello!
>
> We also have this wonderful tool: https://mtcga.gridgain.com/
>
> It keeps track of tests which are currently flaky or under investigation,
> and given your change it will report non-flaky tests that failed, skipping
> all problematic tests.
>
> It talks to TeamCity so it can run tests for you, check their results and
> comment JIRA with "Bot Visa", see:
> https://issues.apache.org/jira/browse/IGNITE-11299 comment from "Ignite TC
> Bot"
>
> Since we don't have too many flaky tests thanks to Make TeamCity Green
> Again movement, it works reasonably well.
>
> A year ago we did not have any of this but now we do, and we were able to
> shave 80% of test-checking busy work. This is mostly thanks to Dmitry
> Pavlov.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 27 февр. 2019 г. в 01:39, Stanislav Kozlovski <
> Stanislav_Kozlovski@outlook.com>:
>
> > Hey there Ignite community,
> >
> > I contribute to a fellow open-source project - Apache Kafka - and there
> we
> > have been fighting flaky tests a lot. We run Java 8 and Java 11 builds on
> > every Pull Request and due to test flakiness, almost all of them turn out
> > red with 1 or 2 tests (completely unrelated to the change in the PR)
> > failing. This has resulted in committers either ignoring them and merging
> > the changes or in the worst case rerunning the hour-long build until it
> > becomes green.
> > This test flakiness has also slowed down our releases significantly.
> >
> > In general, I was just curious to understand if this is a problem that
> > your project faces as well. Does your project have a lot of
> intermittently
> > failing tests, do you have any active process of addressing such tests
> > (during the initial review, after realizing it is flaky, etc). Any
> pointers
> > will be greatly appreciated!
> >
> > Thanks,
> > Stanislav
> >
> >
> > <http://aka.ms/weboutlook>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message