harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stepan Mishura" <stepan.mish...@gmail.com>
Subject [build-and-test] Thoughts about testing infra
Date Tue, 04 Mar 2008 09:39:45 GMT
Hi folks,

I've moved 2.0 build-and-test's branch to the trunk - the branch was
used during M2-M5 testing and IMO it proved to be very useful (also I
tagged the former trunk as 1.0.)

I think that this is a good time to collect ideas and hear opinions
what should be improved and possible directions for test infra

I have to admit that the current testing infra is far from being
perfect – there a lot for work to make is more usable, cleaning up
current scripts, sort out all know issues with tests/scenarios and so
on. But I'd like to address problems I believe of different level. I
have an impression that Harmony tests suites are still in shadow (IOW,
IMHO they are undervalued) of testing automation (i.e. CruiseControl
and support scripts) – and most of folks prefer to contribute to the
code development. And I believe that it is possible to make tests
development also interesting too. IMO the missed point that most of
Harmony test suites can be used as stand alone suites to verify/test
different implementations – in other words, the suites are not tightly
coupled with DRL VM, for example, it is possible to run them against
IBM VME (BTW, may it be makes sense to run full set of suites against
IBM VME for each milestone candidate as for DRL VM. Any volunteers?)

I'm going to update the test infra home page [1] with info related to
the renewed trunk. Also I think the page should contain some words
about project's testing strategy: i.e. how Harmony QA and release
process are organized - integrity/snapshot testing, milestones, code
freeze and etc., what suites are there (their description and
purpose). I think this may encourage people to invest efforts into
developing tests.

Other ideas and thoughts?

[1] http://harmony.apache.org/subcomponents/buildtest/index.html


View raw message