harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Ellison <t.p.elli...@gmail.com>
Subject Re: Test framework
Date Tue, 17 Jan 2006 14:19:49 GMT
Stepan Mishura wrote:
> A test framework is quite essential part and we should define it on early
> stage.
> 
> I think first we should agree on which test we are going to provide and form
> requirements to them. There are no doubts that we will provide unit tests
> for classlib. So unit tests are provided by code authors in JUnit format and
> intended to verify all code functionality. They are placed in the same
> package as tested code and run on the bootclasspath.
> 
> Anything to add? Are there any objections?

You have seen that there *are* objections to putting the test code into
the implementation packages and running it on the bootclasspath.  I can
restate the reasons if it helps.

> Conformance tests we will receive form TCK and what about other test suites:
> functional, stress tests, performance? Will we create a separate framework
> for each?

Running the TCK is a certification step, not a suite of tests that we
develop against to get a progressively compliant implementation.

I believe we need:

- implementation tests : using the 'trick' that Geir refers to below to
expose some internal APIs to the application code.  We would only expect
these to pass on Harmony code.

- API unit tests : driving the published APIs, these should pass on all
compliant implementations (modulo known bugs in those impls.).

- then, as you say, additional 'system', 'stress', 'performance', tests
which are likely to fall outside the unit test framework.

Regards,
Tim

> Thanks,
> Stepan Mishura,
> Intel Middleware Products Division
> 
> 
> On 1/16/06, Geir Magnusson Jr <geir@pobox.com> wrote:
>> One thing that's popped up on the "Test suite layout" thread is the
>> thought that we need to b0rk the canonical package and naming
>> conventions for unit tests in order to be able to run things on the boot
>> classpath of the VM.  I think this issue is important enough and
>> fundamental enough to warrant it's own thread.
>>
>> First, do we really need to do this?  I thought that we (Tim and I) had
>> informally discussed this at ApacheCon and came to some good conclusion
>> where we were able to figure out a trick.
>>
>> Second, shouldn't we think about providing a test environment in which
>> we can completely control the environment  - we can test the class
>> library in a container that can be run in any VM so we have full control
>> over security and other issues?
>>
>> Of course, I'd like both.  If we do have the "trick" that we talked
>> about, then we can use canonical JUnit (or TestNG) naming and package
>> conventions, which I think is important.
>>
>> geir
>>
>>
> 

-- 

Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.

Mime
View raw message