harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Fedotov" <alexei.fedo...@gmail.com>
Subject Re: Is it possibel to use a debug build to run stress tests?
Date Wed, 24 Oct 2007 12:54:16 GMT
Hello, Stepan,

You wrote:
>  I assume that on debug build
> execution time will be ~ 1-2 hours.
Each stress test is designed to run for 30 seconds or less, the
timeout is configured along with other suite parameters. You are
correct assuming that the total time depends on a build configuration
because each test spends some time to warm up and shut down.

> Or do you think that stress suite is a
> special one and it is worth to set up a separate (debug) cycle for it?
Yes, I think so. To some degree the only way to motivate a developer
to fix a stress behavior is to show the crash, and asserts really
help.

Thanks.

On 10/24/07, Stepan Mishura <stepan.mishura@gmail.com> wrote:
> > Is it possibel to use a debug build to run stress tests?
>
> Technically - yes (we are free to run the suite in any way on any
> build), but practically - see below.
>
> On 10/23/07, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
> > Stepan,
> >
> > I believe the test suite run time is not be a factor which affects our
> > decision on a configuration which is used for the test run. Most of
> > stress tests have a duration of 30 seconds, regardless of the build
> > which is used for the testing. The total test suite run time is within
> > 40 minutes.
> >
>
> I don't agree with you about execution time. It was also taken into
> account when testing cycles were planed.
>
> There is "integrity testing cycle" that runs selected suites against
> debug build.
> I encourage everyone to visit regularly integrity testing status
> page[1] to check if your applied patch don't introduce any
> regressions.
> It runs 24x7 and its aim to detect regressions as soon as possible.
> Currently it takes 4-5 hours (depends on host) to complete integrity
> testing cycle and I would make it shorter, if possible. I think it is
> not good to add more suites to this cycle.  BTW, 40 minutes for stress
> suites is time for release build and I assume that on debug build
> execution time will be ~ 1-2 hours.
>
> The stress suite was included to "snapshot testing cycle". Your
> arguments for "validity asserts disabled" (IOW, debug vs release)
> relates to all suites in snapshot testing. As I wrote in previous
> e-mial "if we switch to debug build I'm afraid it will be impossible
> to test 3 snapshots per week.". Or do you think that stress suite is a
> special one and it is worth to set up a separate (debug) cycle for it?
>
> [1] http://people.apache.org/~varlax/harmony-integrity/
>
> Thanks,
> Stepan.
>
> > You argument about testing binaries which are delivered to the
> > customer is pretty valid from my point of view. From the other side it
> > makes little sense to look for root enumeration errors with heap
> > validity asserts disabled. Does anyone see how can we resolve this
> > situation?
> >
> > Thanks.
> >
>


-- 
With best regards,
Alexei,
ESSD, Intel

Mime
View raw message