harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stepan Mishura" <stepan.mish...@gmail.com>
Subject Re: Is it possibel to use a debug build to run stress tests?
Date Wed, 24 Oct 2007 09:46:36 GMT
> Is it possibel to use a debug build to run stress tests?

Technically - yes (we are free to run the suite in any way on any
build), but practically - see below.

On 10/23/07, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
> Stepan,
>
> I believe the test suite run time is not be a factor which affects our
> decision on a configuration which is used for the test run. Most of
> stress tests have a duration of 30 seconds, regardless of the build
> which is used for the testing. The total test suite run time is within
> 40 minutes.
>

I don't agree with you about execution time. It was also taken into
account when testing cycles were planed.

There is "integrity testing cycle" that runs selected suites against
debug build.
I encourage everyone to visit regularly integrity testing status
page[1] to check if your applied patch don't introduce any
regressions.
It runs 24x7 and its aim to detect regressions as soon as possible.
Currently it takes 4-5 hours (depends on host) to complete integrity
testing cycle and I would make it shorter, if possible. I think it is
not good to add more suites to this cycle.  BTW, 40 minutes for stress
suites is time for release build and I assume that on debug build
execution time will be ~ 1-2 hours.

The stress suite was included to "snapshot testing cycle". Your
arguments for "validity asserts disabled" (IOW, debug vs release)
relates to all suites in snapshot testing. As I wrote in previous
e-mial "if we switch to debug build I'm afraid it will be impossible
to test 3 snapshots per week.". Or do you think that stress suite is a
special one and it is worth to set up a separate (debug) cycle for it?

[1] http://people.apache.org/~varlax/harmony-integrity/

Thanks,
Stepan.

> You argument about testing binaries which are delivered to the
> customer is pretty valid from my point of view. From the other side it
> makes little sense to look for root enumeration errors with heap
> validity asserts disabled. Does anyone see how can we resolve this
> situation?
>
> Thanks.
>

Mime
View raw message