ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitry Pavlov <dpavlov....@gmail.com>
Subject Re: Make Teamcity Green Again
Date Tue, 13 Feb 2018 11:29:36 GMT
Forgot to share link to actual investigations list
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&tab=investigations

Please feel free to create and assign investigations if you have assumption
about causes of failure.

вт, 13 февр. 2018 г. в 1:34, Dmitriy Setrakyan <dsetrakyan@apache.org>:

> Dmitriy, thanks for pushing this! We have to bring the number of the
> failing tests to zero. I hope that the community gets behind this effort.
>
> D.
>
> On Mon, Feb 12, 2018 at 8:26 AM, Dmitry Pavlov <dpavlov.spb@gmail.com>
> wrote:
>
> > Hi Folks,
> >
> > I want to resurrect this process and start to fix tests. Currently
> approx.
> > 60-80 tests are failed each run on master.
> >
> > As one from the first point I suggest to use TC investigations for brief
> > research of test failures.
> >
> > If you can help with research please create and assign investigation on
> TC
> > (on test fail select dropdown menu, then 'Investigate / Mute'
> > then 'Investigated by: me' & 'Resolve: auto/manually'; Runs
> > https://ci.ignite.apache.org/viewType.html?buildTypeId=
> > IgniteTests24Java8_RunAll&branch_IgniteTests24Java8=%3Cdefault%3E
> > ). Also help with existing fixes review/merge would be very appreciated.
> >
> > Issue creation is still desired, but investigation will help us to
> identify
> > tests failures which are under research and failures not researched for
> > now.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > вт, 7 нояб. 2017 г. в 0:06, Dmitry Pavlov <dpavlov.spb@gmail.com>:
> >
> > > "Ignored" means that the test is disabled from the code, using
> annotation
> > > @Ignore. Not sure it works for our JUnit3 style Java tests. It is used
> > for
> > > .NET.
> > > "Muted" test is disabled in the interface TeamCity, but test code is
> > > actually executed (which is why hanging tests needs to be failed in
> > > addition to mute).
> > >
> > > The fact that the run all now became green means all the tests that
> fall
> > > at this point are flaky. If it is about 200 non-muted flaky tests
> having
> > > the probability of passing 98%, then the whole chain will pass only
> with
> > > the probability 0.98 ^ 200 = 1.7%
> > >
> > > I suggest continue to identify, fix, or mute such flaky tests to get
> > > stable green RunAll for correct code.
> > >
> > > пн, 6 нояб. 2017 г. в 22:57, Denis Magda <dmagda@apache.org>:
> > >
> > >> Dmitriy,
> > >>
> > >> Now it’s time to fix a pretty decent number of muted and ignored
> tests.
> > >> BTW, what’s the difference between “ignored” and “muted”? I’ve
seen
> the
> > >> former before.
> > >>
> > >> —
> > >> Denis
> > >>
> > >> > On Nov 5, 2017, at 9:58 AM, Dmitry Pavlov <dpavlov.spb@gmail.com>
> > >> wrote:
> > >> >
> > >> > Hi Igniters,
> > >> >
> > >> > I am happy to report that the activity of monitoring and fixing the
> > >> tests
> > >> > brought the first significant victory!
> > >> >
> > >> > Today tests launch running all Ignite tests (Ignite 2.0
> Tests->RunAll)
> > >> is
> > >> > completely green:
> > >> https://ci.ignite.apache.org/viewLog.html?buildId=930953
> > >> >
> > >> > This is an reason for being pride, but not a reason to stop this
> > >> activity.
> > >> > Unfortunately, there are still a number of tests with unpredictable
> > >> > failures (flaky, 234 tests), and a number of tests have been muted.
> > Full
> > >> > project scope is available in the article
> > >> >
> > >> https://cwiki.apache.org/confluence/display/IGNITE/
> > Make+Teamcity+Green+Again
> > >> >
> > >> > In any case, thank to all of you who helped and continues to help
> the
> > >> > project to correct the tests and bugs.
> > >> >
> > >> > Sincerely,
> > >> > Pavlov Dmitry
> > >> >
> > >> >
> > >> > пт, 21 июл. 2017 г. в 14:29, Pavel Tupitsyn <ptupitsyn@apache.org>:
> > >> >
> > >> >>> Green Again
> > >> >>> Again
> > >> >> As if it ever was green :)
> > >> >>
> > >> >> Of course +1 on this, let me know if you see any .NET-related
> > failures.
> > >> >>
> > >> >> On Fri, Jul 21, 2017 at 1:06 PM, Николай Ижиков <
> > >> nizhikov.dev@gmail.com>
> > >> >> wrote:
> > >> >>
> > >> >>> Hello, Igniters.
> > >> >>>
> > >> >>> Also ready to help to #MakeTeamcityGreenAgain !
> > >> >>>
> > >> >>> 21 июля 2017 г. 12:56 PM пользователь "Vyacheslav
Daradur" <
> > >> >>> daradurvs@gmail.com> написал:
> > >> >>>
> > >> >>>> Hi guys.
> > >> >>>>
> > >> >>>> I vote for #MakeTeamcityGreenAgain. :-)
> > >> >>>>
> > >> >>>> FYI: it had been described and supported previously[1]
> > >> >>>>
> > >> >>>> After the completion of my current task I will try to
help with
> > this
> > >> >>>> activity.
> > >> >>>>
> > >> >>>> [1]
> > >> >>>> http://apache-ignite-developers.2346864.n4.nabble.
> > >> >>>> com/Test-failures-td14353.html
> > >> >>>>
> > >> >>>> 2017-07-21 12:39 GMT+03:00 Anton Vinogradov <av@apache.org>:
> > >> >>>>
> > >> >>>>> Nikolay,
> > >> >>>>>
> > >> >>>>> That's also a big problem for me, as reviewer, to
accept changes
> > and
> > >> >>>> merge
> > >> >>>>> them to master.
> > >> >>>>>
> > >> >>>>> Dmitriy,
> > >> >>>>>
> > >> >>>>> Currently we have some contributions *from community*
blocked by
> > red
> > >> >>>>> Teamcity.
> > >> >>>>>
> > >> >>>>> On Fri, Jul 21, 2017 at 12:47 AM, Nikolay Izhikov
<
> > >> >>>> nizhikov.dev@gmail.com>
> > >> >>>>> wrote:
> > >> >>>>>
> > >> >>>>>> +1 to Dmitry Pavlov from me.
> > >> >>>>>>
> > >> >>>>>> Green team city would be very helpful.
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>>> If you are a newcomer, please share your feeling:
are you
> > >> >> confused
> > >> >>> by
> > >> >>>>>> test
> > >> >>>>>>> failures on Teamcity?
> > >> >>>>>>
> > >> >>>>>> I'm working on my very first ignite issue
> > >> >>>>>> and it's not easy to understand TC build results.
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>> 21.07.2017 00:30, Dmitriy Setrakyan пишет:
> > >> >>>>>>
> > >> >>>>>> I think the right question to ask is why do we
have RED tests
> to
> > >> >>> begin
> > >> >>>>>>> with. In my view, TC should be green. If there
are tests that
> do
> > >> >> not
> > >> >>>>> pass,
> > >> >>>>>>> we should disable them and file blocker tickets
for the next
> > >> >>> releases.
> > >> >>>>>>>
> > >> >>>>>>> D.
> > >> >>>>>>>
> > >> >>>>>>> On Thu, Jul 20, 2017 at 1:20 PM, Dmitry Pavlov
<
> > >> >>> dpavlov.spb@gmail.com
> > >> >>>>>
> > >> >>>>>>> wrote:
> > >> >>>>>>>
> > >> >>>>>>> Hi Igniters,
> > >> >>>>>>>>
> > >> >>>>>>>> We all know that there are a lot of open
tasks in Ignite
> > project.
> > >> >>> To
> > >> >>>> be
> > >> >>>>>>>> able to handle all these tasks, we need
to grow our
> community.
> > >> >> And
> > >> >>> to
> > >> >>>>>>>> achieve that, we should pay special attention
to new
> community
> > >> >>>> members.
> > >> >>>>>>>>
> > >> >>>>>>>> As recent newcomer I’ve faced the following
problem: There is
> > no
> > >> >>> way
> > >> >>>> to
> > >> >>>>>>>> find out by yourself whether your pull
request changes are
> > >> >> correct
> > >> >>> or
> > >> >>>>>>>> not.
> > >> >>>>>>>> Contributor can not distinguish pull request
test failures
> and
> > >> >>> master
> > >> >>>>>>>> test
> > >> >>>>>>>> failures. Experienced Ignite contributor
can estimate the
> > damage,
> > >> >>> but
> > >> >>>>> how
> > >> >>>>>>>> it can be done by new community member?
> > >> >>>>>>>>
> > >> >>>>>>>> Once failing test is introduced to master
it is multiplexed
> to
> > >> >> all
> > >> >>>>> later
> > >> >>>>>>>> PRs. As a result, we have tons of failures
and suites
> timeouts,
> > >> >>> which
> > >> >>>>>>>> waste
> > >> >>>>>>>> Teamcity agents time.
> > >> >>>>>>>>
> > >> >>>>>>>> I suggest to start new activity - MakeTeamcityGreenAgain.
I’m
> > >> >> sure
> > >> >>>> that
> > >> >>>>>>>> there are a lot of experienced developers
in the community
> who
> > >> >>>>> understand
> > >> >>>>>>>> issues with tests very quickly. Please
contact me if you are
> > >> >> ready
> > >> >>> to
> > >> >>>>>>>> help
> > >> >>>>>>>> the community to fix current test issues.
> > >> >>>>>>>>
> > >> >>>>>>>> I would appreciate any ideas and sharing
your vision how we
> can
> > >> >>>> achieve
> > >> >>>>>>>> green teamcity.
> > >> >>>>>>>>
> > >> >>>>>>>> If you are a newcomer, please share your
feeling: are you
> > >> >> confused
> > >> >>> by
> > >> >>>>>>>> test
> > >> >>>>>>>> failures on Teamcity?
> > >> >>>>>>>>
> > >> >>>>>>>> Sincerely,
> > >> >>>>>>>>
> > >> >>>>>>>> Dmitriy Pavlov
> > >> >>>>>>>>
> > >> >>>>>>>>
> > >> >>>>>>>
> > >> >>>>>
> > >> >>>>
> > >> >>>>
> > >> >>>>
> > >> >>>> --
> > >> >>>> Best Regards, Vyacheslav D.
> > >> >>>>
> > >> >>>
> > >> >>
> > >>
> > >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message