harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Kleymenov" <kleyme...@gmail.com>
Subject Re: [drlvm][testing][jit] regular testing for JIT regressions on HUT
Date Thu, 28 Dec 2006 12:31:42 GMT
Tim,

On 12/28/06, Tim Ellison <t.p.ellison@gmail.com> wrote:
> Alexander Kleymenov wrote:
> > Stop, guys, let's think. Regression test is a test checking for
> > particular problem reported and fixed before. So regression test and
> > JIRA report have one-to-one correspondence. What I see at HARMONY-1531
> > is a set of tests for Jitrino.OPT marked as "improvement", not as a bug.
> > So, I think, these tests should go to different place, not to regression
> > test suite. Right?
>
> While I understand what a regression test is, can you help me understand
> why you need to treat them differently to the functional unit tests
> written by developers when the originally authored the code?  i.e. they
> all still need to pass, what do you use the regression classification to do?

Regression test suite keeps us from going back to resolved problems,
from quality degradation. It insures us to be on achieved quality level.
While other test suites insures product works as expected by developer.

Sometimes it is needed to be sure that some fix does not roll to back
the product. It could be easily and quickly done by running the
regression test suite. And we don't have to run all the tests – it just
could take a long time.

So with separate regression test suite we can perform fast pre-commit
testing, and/or do ongoing automated code integrity testing (each 2
hours for example).

In our case it is possible to include vm regression tests into CI CC
right now. Other test packages will be included into another CCs.

Thanks,
Alexander

Mime
View raw message