harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Qiu" <sean.xx....@gmail.com>
Subject Re: [build-and-test] Thoughts about testing infra
Date Wed, 05 Mar 2008 16:00:25 GMT
2008/3/4, Stepan Mishura <stepan.mishura@gmail.com>:
> Hi folks,
> I've moved 2.0 build-and-test's branch to the trunk - the branch was
> used during M2-M5 testing and IMO it proved to be very useful (also I
> tagged the former trunk as 1.0.)

Great news. Thanks.

> I think that this is a good time to collect ideas and hear opinions
> what should be improved and possible directions for test infra
> development.

Shall we supply a friendly front-end for user?
List all the suites available and make the configuration more easy.

> I have to admit that the current testing infra is far from being
> perfect ĘC there a lot for work to make is more usable, cleaning up
> current scripts, sort out all know issues with tests/scenarios and so
> on. But I'd like to address problems I believe of different level. I
> have an impression that Harmony tests suites are still in shadow (IOW,
> IMHO they are undervalued) of testing automation (i.e. CruiseControl
> and support scripts) ĘC and most of folks prefer to contribute to the
> code development. And I believe that it is possible to make tests
> development also interesting too. IMO the missed point that most of

Agree :)

> Harmony test suites can be used as stand alone suites to verify/test
> different implementations ĘC in other words, the suites are not tightly
> coupled with DRL VM, for example, it is possible to run them against
> IBM VME (BTW, may it be makes sense to run full set of suites against
> IBM VME for each milestone candidate as for DRL VM. Any volunteers?)

It sounds reasonable, in fact, i am going to plan the same thing.
Maybe we can make our BTI a more flexible infrastrcture for more runtime.
We can pick up a runtime before starting the BTI as well as the test
suite, such as:

drlvm+harmony -|               |- classlib unit test
j9vm+harmony  -| --  BTI ----|- vm unit test
...                    -|               |- ...

Volunteer :)

> I'm going to update the test infra home page [1] with info related to
> the renewed trunk. Also I think the page should contain some words
> about project's testing strategy: i.e. how Harmony QA and release
> process are organized - integrity/snapshot testing, milestones, code
> freeze and etc., what suites are there (their description and
> purpose). I think this may encourage people to invest efforts into
> developing tests.
> Other ideas and thoughts?
> [1] http://harmony.apache.org/subcomponents/buildtest/index.html
> Thanks,
> Stepan.

Sean, Xiao Xia Qiu
China Software Development Lab, IBM
View raw message