ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Павлухин Иван <vololo...@gmail.com>
Subject Re: Brainstorm: Make TC Run All faster
Date Wed, 01 May 2019 05:34:22 GMT
Nikolay,

> I thought by "faster" we all meant time between One push RunAll button and one get results.
> What else definition are possible here?

But this "faster" depends on a current load. E.g. if we have a lot of
idle slaves then incresead parallelism will allow a build to complete
"faster". If we have many slaves busy than parallelism might "slow"
down an execution. I predict that you are mostly interested in average
metrics when TC has a build queue as it usually have on weekdays. If
so then aggregatin other related jobs into "Build Apache Ignite" might
speed execution of RunAll measured in wall clock time.

And also I would like to return to a point I already mentioned earlier
in other conversations. Our CI environment suites perfectly for an
agile way of working. I suppose that we could try different setups
almost freely and see if it gives a desired effect (and rollback if
no). I am ok to go with Vyacheslav proposal. But I would like to
observe some metrics showing that builds become running faster than
before. Can we do this?

вт, 30 апр. 2019 г. в 17:42, Dmitriy Pavlov <dpavlov@apache.org>:
>
> Hi, One more metric we can introduce is efficiency. Test run time/overall
> time. The goal of launching RunAll is to obtain tests results.
>
> I've added more statistics to be displayed in the TC Bot, so now you may
> click 'more' button near 'chain results' caption and you'll see sum of
> statistics for all chain or individual suite. Latest RunAll gives the
> following:
> Overall time 57h 52m 41.188s (
> - Build Net Time: 45h 10m 31.686s,
> - Tests: 38h 58m 30.8s,
> - Src. Update: 2h 29m 51.14s,
> - Artifacts Publishing: 26m 38.295s,
> - Dependencies Resolving: 9h 45m 1.809s,
> - Timeouts: 3h 54m 16.071s)
>
> Currently, Efficiency is around 67-72%
>
> Sincerely,
> Dmitriy Pavlov
>
> пн, 29 апр. 2019 г. в 12:57, Nikolay Izhikov <nizhikov@apache.org>:
>
> > Ivan,
> >
> > I thought by "faster" we all meant time between One push RunAll button and
> > one get results.
> > What else definition are possible here?
> >
> > В Пн, 29/04/2019 в 12:04 +0300, Павлухин Иван пишет:
> > > Nikolay,
> > >
> > > > Why we should imagine it?
> > >
> > > Only in order to understand what do we mean by "faster". My question was:
> > >
> > > 1st approach will take less agent time in sum. 2nd approach will
> > > complete faster in wall clock time. And your main concern is "total
> > > agent time". Am I right?
> > >
> > > пн, 29 апр. 2019 г. в 12:01, Nikolay Izhikov <nizhikov@apache.org>:
> > > >
> > > > > Let's imagine that we have an infinite number of agents
> > > >
> > > > Why we should imagine it?
> > > > We don't have infinite number of agents.
> > > >
> > > > And we have several concurrent Run All.
> > > >
> > > >
> > > > В Пн, 29/04/2019 в 11:50 +0300, Павлухин Иван пишет:
> > > > > Vyacheslav,
> > > > >
> > > > > I finally figured out that "faster" means "total agent time".
> > > > >
> > > > > Let's imagine that we have an infinite number of agents. And 2
> > approaches:
> > > > > 1. Uber "Build Apache Ignite" containing all checks.
> > > > > 2. Separate jobs for compilation, checkstyle and etc.
> > > > >
> > > > > 1st approach will take less agent time in sum. 2nd approach will
> > > > > complete faster in wall clock time. And your main concern is "total
> > > > > agent time". Am I right?
> > > > >
> > > > > пн, 29 апр. 2019 г. в 11:42, Vyacheslav Daradur <daradurvs@gmail.com
> > >:
> > > > > >
> > > > > > Hi, Ivan,
> > > > > >
> > > > > > We are in the thread "Make TC Run All faster", so the main benefit
> > is
> > > > > > to make TC faster :)
> > > > > >
> > > > > > Advantages:
> > > > > > - 1 TC agent required instead of 4;
> > > > > > - RunAll will be faster, in case of builds queue;
> > > > > >
> > > > > > Also, the "licenses" profile is included in the step of a release
> > > > > > build. I believe check-style also should be included.
> > > > > >
> > > > > > The generation of Javadocs is an optional step at preparing
the
> > > > > > release, but its check on TC takes significant time in case
of the
> > > > > > separate build.
> > > > > >
> > > > > > > > Returning to "Build Apache Ignite" it seems to me
that
> > ideally, it can
> > > > > >
> > > > > > be hierarchical.
> > > > > >
> > > > > > I agree, all the checks may be set as a separate step in the
> > build's
> > > > > > configuration. This helps with the main problem I'm trying to
> > solve -
> > > > > > resolving of dependencies which takes the most time of the builds.
> > > > > >
> > > > > > On Mon, Apr 29, 2019 at 11:24 AM Павлухин Иван <
> > vololo100@gmail.com> wrote:
> > > > > > >
> > > > > > > Vyacheslav, Maxim,
> > > > > > >
> > > > > > > Can we once again outline what benefits aggregated "Build
Apache
> > > > > > > Ignite" performing various checks has comparing to a modularized
> > > > > > > approach in which separate builds perform separate tasks?
> > > > > > >
> > > > > > > For example, modularized approach looks nice because it
is
> > similar to
> > > > > > > good practices in software development where we separate
> > > > > > > responsibilities between different classes instead of
> > aggregating them
> > > > > > > into a single class. And as usual multiple classes works
together
> > > > > > > coordinating by a class from upper level. So, in fact it
is a
> > > > > > > hierarchical structure.
> > > > > > >
> > > > > > > Returning to "Build Apache Ignite" it seems to me that
ideally
> > it can
> > > > > > > be hierarchical. There is a top level compilation (assembly?)
> > job but
> > > > > > > it is always clear what tasks does it perform (check style,
check
> > > > > > > license and other subjobs).
> > > > > > >
> > > > > > > пт, 26 апр. 2019 г. в 17:06, Maxim Muzafarov <maxmuzaf@gmail.com
> > >:
> > > > > > > >
> > > > > > > > Folks,
> > > > > > > >
> > > > > > > > +1 for merging all these suites into the single one.
All these
> > suites
> > > > > > > > (Build Apache Ignite, Javadoc, Licenses Header, Checkstyle)
> > required
> > > > > > > > to be `green` all the time. So, we can consider making
them a
> > part of
> > > > > > > > build Apache Ignite procedure.
> > > > > > > >
> > > > > > > > Also, I'd suggest going deeper. We can try to merge
`Licenses
> > Header`
> > > > > > > > into the `Code style checker` [1]. This will simplify
the code
> > > > > > > > checking process.
> > > > > > > >
> > > > > > > > [1] http://checkstyle.sourceforge.net/config_header.html
> > > > > > > >
> > > > > > > > On Fri, 26 Apr 2019 at 13:17, Vyacheslav Daradur <
> > daradurvs@gmail.com> wrote:
> > > > > > > > >
> > > > > > > > > Ivan, you are right, I meant to combine them
into one.
> > > > > > > > >
> > > > > > > > > Here is a build [1], with enabled profiles (check-licenses,
> > > > > > > > > checkstyle) and check of javadoc to show the
idea.
> > > > > > > > >
> > > > > > > > > Seems it takes ~15 minutes.
> > > > > > > > >
> > > > > > > > > [1]
> > https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_ExperimentalBuildApacheIgniteJavadocLicensesHeaderCheckstyle&branch_IgniteTests24Java8=
> > <default>;;
> > > > > > > > >
> > > > > > > > > On Fri, Apr 26, 2019 at 12:06 PM Павлухин
Иван <
> > vololo100@gmail.com> wrote:
> > > > > > > > > >
> > > > > > > > > > Hi Vyacheslav,
> > > > > > > > > >
> > > > > > > > > > What do you mean by uniting?
> > > > > > > > > >
> > > > > > > > > > For me it looks like [Javadocs] and [Check
Code Style] are
> > not so time
> > > > > > > > > > consuming comparing to tests, are not they?
Do you suggest
> > to combine
> > > > > > > > > > mentioned 4 jobs into one? How long will
it run in a such
> > case?
> > > > > > > > > >
> > > > > > > > > > чт, 25 апр. 2019 г. в 10:50, Vyacheslav
Daradur <
> > daradurvs@gmail.com>:
> > > > > > > > > > >
> > > > > > > > > > > Hi Igniters,
> > > > > > > > > > >
> > > > > > > > > > > At the moment we have several separated
test suites:
> > > > > > > > > > > * ~Build Apache Ignite~ _ ~10..20mins
> > > > > > > > > > > * [Javadocs] _ ~10mins
> > > > > > > > > > > * [Licenses Headers] _ ~1min
> > > > > > > > > > > * [Check Code Style] _ ~7min
> > > > > > > > > > > The most time of each build (except
Licenses Headers) is
> > taken by
> > > > > > > > > > > dependency resolving.
> > > > > > > > > > >
> > > > > > > > > > > Their main goal is a check that the
project is built
> > properly.
> > > > > > > > > > >
> > > > > > > > > > > Also, profiles of [Javadocs], [Licenses
Headers] uses at
> > the step of
> > > > > > > > > > > preparing release (see DEVNOTES.txt)
that means they are
> > important.
> > > > > > > > > > >
> > > > > > > > > > > I'd suggest uniting the builds, this
should reduce the
> > time of tests
> > > > > > > > > > > on ~15 minutes and releases agents.
> > > > > > > > > > >
> > > > > > > > > > > What do you think?
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Nov 27, 2018 at 3:56 PM Павлухин
Иван <
> > vololo100@gmail.com> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Roman,
> > > > > > > > > > > >
> > > > > > > > > > > > Do you have some expectations
how faster "correlated"
> > tests
> > > > > > > > > > > > elimination will make Run All?
Also do you have a
> > vision how can we
> > > > > > > > > > > > determine such "correlated" tests,
can we do it
> > relatively fast?
> > > > > > > > > > > >
> > > > > > > > > > > > But all in all, I am not sure
that reducing a group of
> > correlated
> > > > > > > > > > > > tests to only one test can show
good stability.
> > > > > > > > > > > > пн, 26 нояб. 2018 г. в
17:48, aplatonov <
> > aplatonovv@gmail.com>:
> > > > > > > > > > > > >
> > > > > > > > > > > > > It should be noticed that
additional parameter
> > TEST_SCALE_FACTOR was added.
> > > > > > > > > > > > > This parameter with ScaleFactorUtil
methods can be
> > used for test size
> > > > > > > > > > > > > scaling for different runs
(like ordinary and
> > nightly RunALLs). If someone
> > > > > > > > > > > > > want to distinguish these
builds he/she can apply
> > scaling methods from
> > > > > > > > > > > > > ScaleFactorUtil in own tests.
For nightly test
> > TEST_SCALE_FACTOR=1.0, for
> > > > > > > > > > > > > non-nightly builds TEST_SCALE_FACTOR<1.0.
For
> > example in
> > > > > > > > > > > > > GridAbstractCacheInterceptorRebalanceTest
test
> > ScaleFactorUtil was used for
> > > > > > > > > > > > > scaling count of iterations.
I guess that
> > TEST_SCALE_FACTOR support will be
> > > > > > > > > > > > > added to runs at the same
time with RunALL (nightly)
> > runs.
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > > Sent from:
> > http://apache-ignite-developers.2346864.n4.nabble.com/
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > Best regards,
> > > > > > > > > > > > Ivan Pavlukhin
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > Best Regards, Vyacheslav D.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Best regards,
> > > > > > > > > > Ivan Pavlukhin
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Best Regards, Vyacheslav D.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > > Ivan Pavlukhin
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best Regards, Vyacheslav D.
> > > > >
> > > > >
> > > > >
> > >
> > >
> > >
> >



-- 
Best regards,
Ivan Pavlukhin

Mime
View raw message