harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Ellison <t.p.elli...@gmail.com>
Subject Re: [drlvm][testing][jit] regular testing for JIT regressions on HUT
Date Thu, 28 Dec 2006 12:56:13 GMT
Alexander Kleymenov wrote:
> Tim,
> 
> On 12/28/06, Tim Ellison <t.p.ellison@gmail.com> wrote:
>> Alexander Kleymenov wrote:
>> > Stop, guys, let's think. Regression test is a test checking for
>> > particular problem reported and fixed before. So regression test and
>> > JIRA report have one-to-one correspondence. What I see at HARMONY-1531
>> > is a set of tests for Jitrino.OPT marked as "improvement", not as a
>> bug.
>> > So, I think, these tests should go to different place, not to
>> regression
>> > test suite. Right?
>>
>> While I understand what a regression test is, can you help me understand
>> why you need to treat them differently to the functional unit tests
>> written by developers when the originally authored the code?  i.e. they
>> all still need to pass, what do you use the regression classification
>> to do?
> 
> Regression test suite keeps us from going back to resolved problems,
> from quality degradation. It insures us to be on achieved quality level.
> While other test suites insures product works as expected by developer.
> 
> Sometimes it is needed to be sure that some fix does not roll to back
> the product. It could be easily and quickly done by running the
> regression test suite. And we don't have to run all the tests – it just
> could take a long time.
> 
> So with separate regression test suite we can perform fast pre-commit
> testing, and/or do ongoing automated code integrity testing (each 2
> hours for example).
> 
> In our case it is possible to include vm regression tests into CI CC
> right now. Other test packages will be included into another CCs.

I know what regression tests are, I'm trying to understand why you want
keep them separate.  So it's a time thing ... i.e. you would run all the
tests if you had the time?

Tim

Mime
View raw message