harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Ellison <t.p.elli...@gmail.com>
Subject Re: regression test suite
Date Fri, 30 Dec 2005 16:18:49 GMT
Geir Magnusson Jr wrote:
> Thanks for bringing this up.  This has been on my to-do list to talk about.
> 
> inline
> 
> Vladimir Strigun wrote:
> 
>> We already discussed several questions about tests. But I'd like to
>> open a new topic about regression test suite. Some API bugs have been
>> fixed (or patch suggested) and I think it will be useful to have
>> regression tests for all these bugs to avoid them reappearing in the
>> future.
>>
>> I think the natural place for the regression tests should be together
>> with the unit tests. I am not sure, however, if the regression tests
>> should be marked explicitly or differ from the unit tests in any other
>> specific way.
>>
>> What do you think about it? Any preferences?
> 
> 
> I think that putting them in a parallel tree is worth considering, just
> for the sake of organization, as long as this is transparent to the
> tools a developer/user would use to do run the tests.  I do think that
> "regression test" is a broad term, and we might want to harvest things
> out of regression tests for where our unit tests fell short.

Aren't we getting a bit ahead of ourselves here, the existing tests
hardly need organizing (all three of them!) -- but let's go mad and
assume we have lots of tests housed in our repository.

What is the useful distinction for regression tests being kept separate?
 I can see that you may distinguish unit and 'system-level' tests just
because of the difference in frameworks etc. required, but why do I care
if the test was written due to a JIRA issue or test-based development or
someone who get's kicks out of writing tests to break the code?

> I also believe we should be encourging bug reporters to submit a test
> that demonstrates the bug.  We'd acquire a good set of tests at that point.

Agreed, a simple failing test with a description of expected results is
the ideal way to report a bug.  In particular, people are not expected
to suggest a patch when they raise issues, though they are of course
welcome to do so.

> I was going to take a hard look at the build/test framework this
> week(end).  Might as well start now.  Currently, it's JUnit.  I'm
> wondering if anyone would consider switching to TestNG.  I've never used
> it, but it's written by a guy I really respect and the point was to
> address shortcomings he found in JUnit...

I've never used TestNG, so have no opinion at the moment.  What's so
good about it?

Regards,
Tim

> geir
> 
>>
>> Thank you,
>> Vladimir Strigun, Intel Middleware Products Division.
>>
>>
> 

-- 

Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.

Mime
View raw message