harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Kleymenov" <kleyme...@gmail.com>
Subject [buildtest] Proposal for Build Test Infrastructure Improvement
Date Tue, 27 Mar 2007 06:43:57 GMT

With this letter I'd like to announce the new approach for BT Framework
Implementation. It could be found as an attachment to JIRA #3501 [1].

Proposed BT Infra Implementation improves and structures current one and brings
the following features allowing using the framework as a standard means for
Harmony Testing works:

  - Simplicity and convenience of use allows to perform all of the testing
    works (by testers and developers) by unified single way and with using
    of the same means.

  - Clear and Documented Architecture, Workspace Structure, Clear separation
    of Functional Components of the Framework, Standard Implementation Means
    allows to easy start, use, improve, and maintain the framework.

  - Highly configurable Test Run:

    Simple means for Test Suite selections - use property value to select
    required Test Suites. Only selected suites will be checked out (or
    downloaded) and executed.

    Simple and easy manageable means for settings dependencies between Test
    Suites. It excludes the necessity of combo-test-suites creation (such as
    classlib-drlvm now)

    Simple and natural means for describing of required parameters for selected
    Test Suites (as path to tested JVM, Classlib, compiler checks, and so on).
    It is done in one specified place and there is no need to drill into Test
    Suites for setting up such a parameters. Moreover, new BT Framework performs
    the correctness analysis of the provided values for parameters and reports
    an error just before Test Suite execution, not after an hour of the work.
    Additional important feature is an ability to provide default values for
    required parameters on the base of execution results of another Test Suite.
    So, for example, drlvm Test Suite (which checks that DRLVM can be built)
    uses the default value for 'classlib.trunk' parameter (specifying
the path to
    classlib workspace) the value provided by classlib Test Suite. If classlib
    Test Suite was chosen for execution, 'classlib.trunk' parameter of drlvm
    test suite will be resolved. If classlib was not chosen, user will be
    asked to provide the value for this parameter. Such a feature allows to
    reduce the user's job on parameters setting.

    Such a simplicity of Test Run configuration (selection of the suites and
    parameters tuning) allows to easy set up the BT Framework for specific
    needs. So for Harmony developers it is possible to chose their own sets of
    checks of the workspace under development and to run this checks on demand
    in single execution mode. For Harmony testers it is possible to use
    sophisticated CruiseControl configurations for continuous test runs with
    notification publishing.

  - Easy and Documented way to integrate new Test Suites:

    There is no longer need to implement CruiseControl configurations for
    each of new Test Suites being integrated into framework. Cruise Control
    system is used as a framework extension allowing continuous test runs, not
    as a development means. It simplifies the process and reduces the time
    of new Test Suite integration.

  - Implemented Approach to share External Library Dependencies
between different
    Test Suites allows to perform external dependencies fetching in one place by
    one means and to save time and internet traffic.

  - To run custom configuration there is no need to download
    all of the Test Suites integrated into BT.

  - The Architecture of the Framework is open for extending by other functional
    parts and 3rd party tools (such as CruiseControl etc)

The main idea of the new Framework is to use Ant's scripts called 'adaptors' as
a glue layer between Test Suites and the BT Framework executing them. All of the
Parameters of the suites (such as required values, external libraries, shared
values to be used by other suites, optional parameters for low-level tuning)
are described in 'parameters.xml' file placed near corresponding 'adaptor.xml'.
That's all.

Please, look at the archive. There is a README.txt and SPEC.txt files describing
and documenting the new Infrastructure. Also there are two integrated Test
Suites - classlib and drlvm which check that classlib and drlvm can be built.
Please look at them under adapters directory to catch the main idea of the

Of course there are many points for improvements, but the main idea and
functionality has already been implemented. Sorry for such a bulk, but I could
not provide the framework by 'logical parts' because they together forms a
whole. Now, when all of them in one place it is possible to improve or
reimplement them (I heard about the existing approach for external libraries
sharing, but did not look at it.  Probably it is better then proposed in my BT
Infra implementation. In this case it is possible to replace the implementation
of this functional part of the Framework).

If you have any questions regarding the implementation please,
provide them into this mail-thread.

Thank you,

[1] http://issues.apache.org/jira/browse/HARMONY-3501

View raw message