ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Setrakyan <dsetrak...@apache.org>
Subject Re: Interesting statistics for our tests
Date Fri, 16 Dec 2016 21:04:30 GMT
In addition to optimizing the test suites, should we add more TC agents as
well?

On Fri, Dec 16, 2016 at 7:51 AM, Vladimir Ozerov <vozerov@gridgain.com>
wrote:

> And one more thing - how much time is spent on duplicated "Binary" suites?
> *15
> hours*, or almost *40%* of total "Run All" execution time. It means, that
> if move suites for *OptimizedMarshaller *(which we consider deprecated)
> into a separate circuit (which will be executed not per-branch, but e.g.
> nightly) we will almost double peak TeamCity throughput.
>
> On Fri, Dec 16, 2016 at 6:23 PM, Vladimir Ozerov <vozerov@gridgain.com>
> wrote:
>
> > Igniters,
> >
> > I collected several thought-provoking numbers for our TeamCity
> > (config-variations tests are excluded):
> >
> > * Number of Java suites: 78
> > * Total execution time without build overhead: 38.5 hours
> > * Total test classes: 3376
> > * Total tests: 60681
> >
> > * 25% of time is consumed by 881 tests (41 test class)
> > * 50% of time is consumed by 2755 tests (153 test class)
> > * 75% of time is consumed by 15911 tests (443 test class)
> >
> > Or to be more demonstrative:
> > * 1.5% of tests consume 25% of time
> > * 4.5% of tests consume 50% of time
> > * 26.2% tests consume 75% of time
> >
> > TOP 3 test classes:
> > * CacheSerializableTransactionsTest - 46 minutes
> > * IgniteCacheCrossCacheTxFailoverTest - 30 minutes
> > * GridCachePartitionedNodeRestartTest - 25 minutes
> >
> > Morale:
> > 1) We have lots of poor failover/concurrency tests requiring
> > disproportional time to run;
> > 2) If we split all tests into "core" and "failover/concurrency" parts
> > properly, then *core* part will take *~15 mins* to answer you with high
> > level of confidence whether your change broken anything or not.
> >
> > Thoughts?
> >
> > Vladimir.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message