ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Павлухин Иван <vololo...@gmail.com>
Subject Re: Brainstorm: Make TC Run All faster
Date Mon, 29 Apr 2019 08:50:06 GMT
Vyacheslav,

I finally figured out that "faster" means "total agent time".

Let's imagine that we have an infinite number of agents. And 2 approaches:
1. Uber "Build Apache Ignite" containing all checks.
2. Separate jobs for compilation, checkstyle and etc.

1st approach will take less agent time in sum. 2nd approach will
complete faster in wall clock time. And your main concern is "total
agent time". Am I right?

пн, 29 апр. 2019 г. в 11:42, Vyacheslav Daradur <daradurvs@gmail.com>:
>
> Hi, Ivan,
>
> We are in the thread "Make TC Run All faster", so the main benefit is
> to make TC faster :)
>
> Advantages:
> - 1 TC agent required instead of 4;
> - RunAll will be faster, in case of builds queue;
>
> Also, the "licenses" profile is included in the step of a release
> build. I believe check-style also should be included.
>
> The generation of Javadocs is an optional step at preparing the
> release, but its check on TC takes significant time in case of the
> separate build.
>
> >> Returning to "Build Apache Ignite" it seems to me that ideally, it can
> be hierarchical.
>
> I agree, all the checks may be set as a separate step in the build's
> configuration. This helps with the main problem I'm trying to solve -
> resolving of dependencies which takes the most time of the builds.
>
> On Mon, Apr 29, 2019 at 11:24 AM Павлухин Иван <vololo100@gmail.com>
wrote:
> >
> > Vyacheslav, Maxim,
> >
> > Can we once again outline what benefits aggregated "Build Apache
> > Ignite" performing various checks has comparing to a modularized
> > approach in which separate builds perform separate tasks?
> >
> > For example, modularized approach looks nice because it is similar to
> > good practices in software development where we separate
> > responsibilities between different classes instead of aggregating them
> > into a single class. And as usual multiple classes works together
> > coordinating by a class from upper level. So, in fact it is a
> > hierarchical structure.
> >
> > Returning to "Build Apache Ignite" it seems to me that ideally it can
> > be hierarchical. There is a top level compilation (assembly?) job but
> > it is always clear what tasks does it perform (check style, check
> > license and other subjobs).
> >
> > пт, 26 апр. 2019 г. в 17:06, Maxim Muzafarov <maxmuzaf@gmail.com>:
> > >
> > > Folks,
> > >
> > > +1 for merging all these suites into the single one. All these suites
> > > (Build Apache Ignite, Javadoc, Licenses Header, Checkstyle) required
> > > to be `green` all the time. So, we can consider making them a part of
> > > build Apache Ignite procedure.
> > >
> > > Also, I'd suggest going deeper. We can try to merge `Licenses Header`
> > > into the `Code style checker` [1]. This will simplify the code
> > > checking process.
> > >
> > > [1] http://checkstyle.sourceforge.net/config_header.html
> > >
> > > On Fri, 26 Apr 2019 at 13:17, Vyacheslav Daradur <daradurvs@gmail.com>
wrote:
> > > >
> > > > Ivan, you are right, I meant to combine them into one.
> > > >
> > > > Here is a build [1], with enabled profiles (check-licenses,
> > > > checkstyle) and check of javadoc to show the idea.
> > > >
> > > > Seems it takes ~15 minutes.
> > > >
> > > > [1] https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_ExperimentalBuildApacheIgniteJavadocLicensesHeaderCheckstyle&branch_IgniteTests24Java8=<default>
> > > >
> > > > On Fri, Apr 26, 2019 at 12:06 PM Павлухин Иван <vololo100@gmail.com>
wrote:
> > > > >
> > > > > Hi Vyacheslav,
> > > > >
> > > > > What do you mean by uniting?
> > > > >
> > > > > For me it looks like [Javadocs] and [Check Code Style] are not so
time
> > > > > consuming comparing to tests, are not they? Do you suggest to combine
> > > > > mentioned 4 jobs into one? How long will it run in a such case?
> > > > >
> > > > > чт, 25 апр. 2019 г. в 10:50, Vyacheslav Daradur <daradurvs@gmail.com>:
> > > > > >
> > > > > > Hi Igniters,
> > > > > >
> > > > > > At the moment we have several separated test suites:
> > > > > > * ~Build Apache Ignite~ _ ~10..20mins
> > > > > > * [Javadocs] _ ~10mins
> > > > > > * [Licenses Headers] _ ~1min
> > > > > > * [Check Code Style] _ ~7min
> > > > > > The most time of each build (except Licenses Headers) is taken
by
> > > > > > dependency resolving.
> > > > > >
> > > > > > Their main goal is a check that the project is built properly.
> > > > > >
> > > > > > Also, profiles of [Javadocs], [Licenses Headers] uses at the
step of
> > > > > > preparing release (see DEVNOTES.txt) that means they are important.
> > > > > >
> > > > > > I'd suggest uniting the builds, this should reduce the time
of tests
> > > > > > on ~15 minutes and releases agents.
> > > > > >
> > > > > > What do you think?
> > > > > >
> > > > > > On Tue, Nov 27, 2018 at 3:56 PM Павлухин Иван <vololo100@gmail.com>
wrote:
> > > > > > >
> > > > > > > Roman,
> > > > > > >
> > > > > > > Do you have some expectations how faster "correlated" tests
> > > > > > > elimination will make Run All? Also do you have a vision
how can we
> > > > > > > determine such "correlated" tests, can we do it relatively
fast?
> > > > > > >
> > > > > > > But all in all, I am not sure that reducing a group of
correlated
> > > > > > > tests to only one test can show good stability.
> > > > > > > пн, 26 нояб. 2018 г. в 17:48, aplatonov <aplatonovv@gmail.com>:
> > > > > > > >
> > > > > > > > It should be noticed that additional parameter TEST_SCALE_FACTOR
was added.
> > > > > > > > This parameter with ScaleFactorUtil methods can be
used for test size
> > > > > > > > scaling for different runs (like ordinary and nightly
RunALLs). If someone
> > > > > > > > want to distinguish these builds he/she can apply
scaling methods from
> > > > > > > > ScaleFactorUtil in own tests. For nightly test TEST_SCALE_FACTOR=1.0,
for
> > > > > > > > non-nightly builds TEST_SCALE_FACTOR<1.0. For example
in
> > > > > > > > GridAbstractCacheInterceptorRebalanceTest test ScaleFactorUtil
was used for
> > > > > > > > scaling count of iterations. I guess that TEST_SCALE_FACTOR
support will be
> > > > > > > > added to runs at the same time with RunALL (nightly)
runs.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > > Ivan Pavlukhin
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best Regards, Vyacheslav D.
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Ivan Pavlukhin
> > > >
> > > >
> > > >
> > > > --
> > > > Best Regards, Vyacheslav D.
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
>
>
>
> --
> Best Regards, Vyacheslav D.



-- 
Best regards,
Ivan Pavlukhin

Mime
View raw message