ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: Interesting statistics for our tests
Date Fri, 16 Dec 2016 15:51:40 GMT
And one more thing - how much time is spent on duplicated "Binary" suites? *15
hours*, or almost *40%* of total "Run All" execution time. It means, that
if move suites for *OptimizedMarshaller *(which we consider deprecated)
into a separate circuit (which will be executed not per-branch, but e.g.
nightly) we will almost double peak TeamCity throughput.

On Fri, Dec 16, 2016 at 6:23 PM, Vladimir Ozerov <vozerov@gridgain.com>
wrote:

> Igniters,
>
> I collected several thought-provoking numbers for our TeamCity
> (config-variations tests are excluded):
>
> * Number of Java suites: 78
> * Total execution time without build overhead: 38.5 hours
> * Total test classes: 3376
> * Total tests: 60681
>
> * 25% of time is consumed by 881 tests (41 test class)
> * 50% of time is consumed by 2755 tests (153 test class)
> * 75% of time is consumed by 15911 tests (443 test class)
>
> Or to be more demonstrative:
> * 1.5% of tests consume 25% of time
> * 4.5% of tests consume 50% of time
> * 26.2% tests consume 75% of time
>
> TOP 3 test classes:
> * CacheSerializableTransactionsTest - 46 minutes
> * IgniteCacheCrossCacheTxFailoverTest - 30 minutes
> * GridCachePartitionedNodeRestartTest - 25 minutes
>
> Morale:
> 1) We have lots of poor failover/concurrency tests requiring
> disproportional time to run;
> 2) If we split all tests into "core" and "failover/concurrency" parts
> properly, then *core* part will take *~15 mins* to answer you with high
> level of confidence whether your change broken anything or not.
>
> Thoughts?
>
> Vladimir.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message