ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Setrakyan <dsetrak...@apache.org>
Subject Re: Ignite Benchmarking Rules
Date Thu, 14 Sep 2017 20:27:31 GMT
Alexey, I completely agree. However, for the benchmarks to be useful, then
need to be run on the same hardware all the time. Apache Ignite does not
have servers sitting around, available to run the benchmarks.

Would be nice to see how other projects address it. Can Amazon donate
servers for the Apache projects?

D.

On Thu, Sep 14, 2017 at 6:25 AM, Aleksei Zaitsev <ignite@alexzaitzev.pro>
wrote:

> Hi, Igniters.
>
> Recently I’ve done some research in benchmarks for Ignite, and noticed
> that we don’t have any rules for running benchmarks and collecting result
> from them. Although sometimes we have tasks, which results need to be
> measured. I propose to formalize such things as:
>  * set of benchmarks,
>  * parameters of launching them,
>  * way of result collection and interpretation,
>  * Ignite cluster configuration.
>
> I don’t think that we need to run benchmarks before every merge into
> master, but in some cases it should be mandatory to compare new results
> with reference values to be sure that changes do not lead to the
> performance degradation.
>
> What do you think?
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message