harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Ivanov" <ivavladi...@gmail.com>
Subject [testing][cruise-control] how testing infrastructure should be improved
Date Fri, 26 Jan 2007 11:50:49 GMT
Hello everyone,

while we have more or less stable CC it is time to do work to extend Harmony
testing :)
Yes, I know that we have some gap in stability on x86_64 platform and on Red
Hat Linux so CC runs tests in the 'perTest' mode but anyway I want to
summarize our discussions about testing and continue extend testing
capability.



The current buildtest infrastructure includes only one case for testing:
cyclic run of Harmony pre-integration tests. Also we have jira issues with
iterative runs, eclipse "hello world" and JEDIT scenarios. And community
members time to time runs applications and report their execution status.



The idea was to collect all test-run scripts into buildtest directory and
implement configurable CC.

Created jira should be integrated to the buildtest infrastructure and
buildtest module should be extended by different application scenario tests,
reliability tests etc. In general case we will have a long CC cycle (up to 1
week J ) and results of this runs should be processed with other procedure
(for example, results should be upload to harmonytest.org and than jira
issues should be created for all failures).


It requires some 'standard' interface for all integrated scripts. I like
classlib interface so how about:
  - call of "ant setup;ant" will run all available scripts;
  - call of "ant -Dmodule=hit setup;ant" will run current version of CC –
Harmony integration tests;
  - call of "ant -Dmodule=eut setup;ant" will run Eclipse  unit test etc.

Note, in this case each module should implement proper 'setup' target and
has configuration for CC. The root-script will iterate over all modules to
call their 'setup' and this setup should include whole test setup
(downloading software, adding modules cc-configuration to working
configuration etc).

Is it OK?



If nobody objects I'll start restructuring of buildtest module and will try
to integrate one from extensions.



 Thanks, Vladimir



PS I think the resulting structure should be easy to extend and may looks
like this:

buildtest

 |--config  (default CC configuration to build classlib and DRLVM)

 |--hit (CC configuration to run Harmony classlib&DRLVM tests)

 |--eclipse

     |-- eut (setup and CC configuration to run eclipse non-interactive
tests)

     |-- eclipse3.1.1

         |-- some scenario

|-- build.xml (common setup + call of module's 'setup')

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message