harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Ellison <t.p.elli...@gmail.com>
Subject Re: [build-and-test] Thoughts about testing infra
Date Wed, 05 Mar 2008 17:13:43 GMT
Stepan Mishura wrote:
> Hi folks,
> I've moved 2.0 build-and-test's branch to the trunk - the branch was
> used during M2-M5 testing and IMO it proved to be very useful (also I
> tagged the former trunk as 1.0.)


> I think that this is a good time to collect ideas and hear opinions
> what should be improved and possible directions for test infra
> development.
> I have to admit that the current testing infra is far from being
> perfect – there a lot for work to make is more usable, cleaning up
> current scripts, sort out all know issues with tests/scenarios and so
> on. But I'd like to address problems I believe of different level. I
> have an impression that Harmony tests suites are still in shadow (IOW,
> IMHO they are undervalued) of testing automation (i.e. CruiseControl
> and support scripts) – and most of folks prefer to contribute to the
> code development. And I believe that it is possible to make tests
> development also interesting too. IMO the missed point that most of
> Harmony test suites can be used as stand alone suites to verify/test
> different implementations – in other words, the suites are not tightly
> coupled with DRL VM, for example, it is possible to run them against
> IBM VME (BTW, may it be makes sense to run full set of suites against
> IBM VME for each milestone candidate as for DRL VM. Any volunteers?)
> I'm going to update the test infra home page [1] with info related to
> the renewed trunk. Also I think the page should contain some words
> about project's testing strategy: i.e. how Harmony QA and release
> process are organized - integrity/snapshot testing, milestones, code
> freeze and etc., what suites are there (their description and
> purpose). I think this may encourage people to invest efforts into
> developing tests.
> Other ideas and thoughts?

Thanks for your investment in the test suites, and taking time to think 
about the improvements.

The thing I struggle with is understanding the current system well 
enough to make a valuable contribution.

It would be really good to have a newbies' guide to the test suites 
document on the website with step-by-step instructions (like the quick 
help for contributors document).

While the architecture documentation would be interesting, I think many 
people work by running the simple case, then get dragged into 
investigating the complex cases when their needs require it.


> [1] http://harmony.apache.org/subcomponents/buildtest/index.html
> Thanks,
> Stepan.

View raw message