brooklyn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Bouron <thomas.bou...@cloudsoftcorp.com>
Subject Re: Disable non-deterministic failing unit tests, while fixing them [BROOKLYN-272]
Date Tue, 24 May 2016 15:56:51 GMT
Hi Aled.

Sorry about this, I overlooked the changes on GitHub and thought you
commented out all the tests, didn't realise it was only... genuine comments!
You can ignore my comment/mail.

On Tue, 24 May 2016 at 15:52 Aled Sage <aled.sage@gmail.com> wrote:

> Hi Thomas,
>
> Not sure I follow. We aren't saying commenting out the tests. We are
> proposing to use a test group to mark them as not included in the tests
> run by `mvn clean install`. It achieves that based on some logic in the
> pom that excludes various groups from different profiles. If you run it
> in your IDE, it would still be run (because we're not using
> `enabled=false`).
>
> Is your suggestion to use enabled=false motivated by a desire to disable
> it in the IDE as well? Or something else?
>
> Aled
>
>
> On 24/05/2016 10:04, Thomas Bouron wrote:
> > Hi Aled.
> >
> > +1 for the marking and disabling those tests. However, I would use the
> > `enabled=false` parameter[1] of the `@Test` annotation instead of
> commented
> > them out.
> >
> > [1] http://www.mkyong.com/unittest/testng-tutorial-3-ignore-test/
> >
> > On Tue, 24 May 2016 at 09:54 Aled Sage <aled.sage@gmail.com> wrote:
> >
> >> Hi all,
> >>
> >> TL;DR: let's disable non-deterministic failing unit tests while they are
> >> being fixed.
> >>
> >> ---
> >>
> >> We have a small number of non-deterministic test failures that are
> >> causing false-negatives when jenkins builds master or builds a PR
> branch.
> >>
> >> Clearly we should fix these tests (either re-writing them to avoid
> >> non-determinism, or fixing whatever underlying bug it is exposing).
> >>
> >> However, I think we need a pragmatic short-term solution to get our
> >> jenkins build being reliable+believable asap. We've been living with the
> >> occasional jenkins build failure for far too long!
> >>
> >> I've created:
> >>
> >>    * https://issues.apache.org/jira/browse/BROOKLYN-272
> >>    * https://github.com/apache/brooklyn-server/pull/147
> >>
> >> The issue tracks the failing tests, and the PR disables them, marking
> >> them in testng as:
> >>
> >>      @Test(groups={"WIP", "Non-deterministic-failure"})
> >>
> >> Thoughts?
> >>
> >> Aled
> >>
> >>
> >> --
> > Thomas Bouron • Software Engineer @ Cloudsoft Corporation •
> > http://www.cloudsoftcorp.com/
> > Github: https://github.com/tbouron
> > Twitter: https://twitter.com/eltibouron
> >
>
> --

Thomas Bouron • Software Engineer @ Cloudsoft Corporation •
http://www.cloudsoftcorp.com/
Github: https://github.com/tbouron
Twitter: https://twitter.com/eltibouron

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message