harmony-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From var...@apache.org
Subject svn commit: r562065 - /harmony/enhanced/buildtest/branches/2.0/dev-guide.txt
Date Thu, 02 Aug 2007 10:07:46 GMT
Author: varlax
Date: Thu Aug  2 03:07:45 2007
New Revision: 562065

URL: http://svn.apache.org/viewvc?view=rev&rev=562065
Log:
Additional document for HARMONY-4293

Added:
    harmony/enhanced/buildtest/branches/2.0/dev-guide.txt   (with props)

Added: harmony/enhanced/buildtest/branches/2.0/dev-guide.txt
URL: http://svn.apache.org/viewvc/harmony/enhanced/buildtest/branches/2.0/dev-guide.txt?view=auto&rev=562065
==============================================================================
--- harmony/enhanced/buildtest/branches/2.0/dev-guide.txt (added)
+++ harmony/enhanced/buildtest/branches/2.0/dev-guide.txt Thu Aug  2 03:07:45 2007
@@ -0,0 +1,591 @@
+==============================================================================
+                              BTI Design Document
+==============================================================================
+:Title:   BTI Design Document - a Guide for BTI Developers
+:Author:  Alexander Kleymenov <kleymenov{ at }gmail{ . }com>
+:Date:    2007/07/05
+
+This document contains detailed description of each of the functional parts
+of Harmony BTI. It could be used as a *Guide for BTI Developers*. 
+For quick start with BTI please look at `README`_ file.
+
+Architectural parts of BTI covered in this document can be summarized
+as follows:
+
+:`Execution Core`_:
+    Maintains BTI configuration and Tests execution.
+    Acts as a glue for other components of BTI.
+    Provides some auxiliary functionality for implementation of 
+    test suites adaptors.
+
+:`Test Suite Adaptors`_:
+    The Ant scripts integrating the test suites to BTI by means of
+    implementation *adapting* interface.
+    Consist of ``adaptor.xml`` file maintaining test execution
+    and ``parameters.xml`` file describing configuration parameters.
+
+:`CTE Module`_:
+    Performs Continuous Test Execution.
+    Is built on the base of CruiseControl system.
+
+:`Execution Status Notifier`_:
+    Sends mail alerts with the status of test execution.
+
+:`External Dependencies Fetching Module`_:
+    Provides the unified means for description of external dependencies
+    used by test suites or BTI.
+    Downloads and installs the external libraries.
+    Allows sharing of external libraries between test suites.
+
+
+Execution Core
+==============
+
+Implemented by the following source files::
+
+    ./build.xml
+    ./buildtest.bat
+    ./buildtest.sh
+    ./scripts/exec-adaptor.xml
+    ./scripts/framework.xml
+    ./scripts/main.xml
+
+This part of BTI is responsible for BTI configuration and execution.
+It acts as an execution engine gluing all other functional parts of
+BTI together. It also provides some functionality supporting the development
+of Test Suite Adaptors.
+
+
+BTI Configuration
+-----------------
+
+The BTI configuration is made by means of the property files.
+The first properties file::
+
+    ./scripts/framework.properties 
+
+contains the properties affecting internals of BTI and *SHOULD NOT*
+be modified by user. The second file::
+
+    ./scripts/parameters.xml
+
+is an XML Ant properties file containing the configuration parameters for all
+of the functional parts of BTI. These parameters can be configured by user
+through the third configuration file::
+
+    ./framework.local.properties
+
+By means of this file user can also tune some test suites' specific parameters
+defined in:: 
+
+    ./adaptors/*/parameters.xml
+
+files.
+
+The last configuration file is::
+
+    ./required-parameters.properties
+
+It is generated by BTI and contains the assignments of parameters *required* by
+test suites selected for execution (more on this in `Test Suite Adaptors`_
+section).
+
+
+BTI Execution
+-------------
+
+BTI execution is launched by means of a command line UI provided by the Core.
+Currently this UI consists of the following Ant targets expected from user as
+parameters to BTI command line scripts (``buildtest.bat`` for MS Windows[x]_,
+and ``buildtest.sh`` for Linux):
+    
+``install``
+    Does an installation of BTI on local workstation from remote SVN
+    repository.
+    Checks out (for initial execution) or updates (for consequent ones)
+    BTI implementation files (``./scripts`` directory) and test suite
+    adaptors (``./adaptors`` directory).
+    This target *MUST* be run at first after initial BTI checkout (see
+    ``./README.txt`` for how to get BTI from SVN repository).
+    Generates ``./framework.local.properties`` property file for local BTI
+    configuration.
+
+``framework-update``
+    Does an update of current BTI installation from the SVN repository.
+    Updates BTI implementation scripts and test suite adaptors.
+
+``setup``
+    Does a setup of the test run. Expects the ``test.suites`` property 
+    to be set to comma separated list of test suites selected for execution.
+    Produces 3 files:
+
+    ``./required-parameters.properties``
+        property file containing the assignments of parameters required 
+        by selected test suites.
+
+    ``./selected-suites.properties``
+        property file containing the  test run configuration. Describes the
+        order in which the selected suites will be run, and dependencies
+        between them.
+
+    ``./resources.location``
+        property file describing the locations of external resources needed by
+        configured test run.
+
+``run``
+    Launches test execution. The selected test suites will be executed in
+    order defined by ``test.suites.names.list`` property in 
+    ``./selected-suites.properties`` file. 
+    At first, BTI executes ``setup`` targets of adaptor scripts of all
+    of the selected test suites (in defined order). If ``setup`` fails for
+    some of the suite, this suite won't be ``run``, and all dependent test
+    suites won't be ``setup`` and ``run``. BTI will warn about all of
+    the failed and skipped test suites by means of its log messages.
+    At second, after successful ``setup``, BTI launches ``run`` targets of
+    selected test suites. Similarly to ``setup`` BTI will skip the execution
+    of dependent test suites if their superior test suites failed to
+    ``run``. The information about failed and skipped test suites will be
+    printed out by BTI to its log.
+    After all runs of the selected test suites BTI ends with *SUCCESS* if
+    there were no test suites failures, and *ERROR* if some suites could not
+    be set up or failed to run.
+
+``run-cc``
+    Does the same as ``run``, but test cycle is continuously repeated. Each
+    iteration of test suites execution is triggered whether by 
+    modifications happened in some of the controlled by test suites SVN 
+    repositories, or by schedule provided by user. See `CTE Module`_ for more 
+    information on Continuous Test Execution.
+
+``clean``
+    Consequently launches ``clean`` targets of adaptors of selected test 
+    suites.
+
+``test-mail``
+    Sends test mail notification. Used to check the operability of the
+    current configurations of `Execution Status Notifier`_ module.
+        
+The execution of BTI can be configured by the following properties specified
+in ``./framework.local.properties`` file::
+
+    framework.parameters.optional.dolog
+    framework.parameters.repository.noupdate
+    framework.parameters.repository.awaited
+
+For descriptions of these parameters look in ``./scripts/parameters.xml`` file.
+
+
+
+Test Suite Adaptors
+===================
+
+A Test Suite Adaptor consists of two Ant files placed under 
+``./adaptors/${suite.name}`` directory:
+    
+adaptor.xml
+-----------
+
+Regular Ant project file implementing 3 targets: ``setup``, ``run``, and
+``clean``:
+
+    **setup**
+        has to perform all the works on setting of the test suite on the
+        local workstation: it should download and build the sources for test
+        suite, check required prerequisites, and so on - everything that
+        should be done before first run of the test suite on the workstation.
+        `Execution Core`_ provides some functionality which can be reused
+        here:
+
+        ``<svn-co url="${url}" trunk="${trunk}"/>`` - macrodefinition checking
+        out the SVN repository located at ``${url}`` to directory pointed by
+        ``${trunk}``.
+
+        ``<svn-update-bt path="${path}"/>`` - macrodefinition updating the
+        local ``${path}`` of BTI repository. It is useful when the sources of
+        the test suite are located under BTI's SVN repository.
+
+    **run**
+        performs the actual run of the test suite. It should "``<fail>``"
+        if test suite's execution status is FAILED. 
+
+    **clean**
+        cleans the products of test suite's execution
+        
+The following properties provided by `Execution Core`_ can be useful for
+implementing these targets:
+
+    :``suite.name``: 
+        holding the name of the suite
+
+    :``suite.adaptor.dir``: 
+        the directory where adaptor is placed
+
+    :``test.sources.dir``: 
+        the root directory containing the test sources.
+        Usually the test sources for each particular test suite are placed
+        under ``${test.sources.dir}/${suite.name}`` directory.
+
+    :``temp.files.dir``: 
+        the directory for temporal files
+
+    :``checkouts.dir``: 
+        the directory for external checkouts. For example *classlib* test
+        suite checks out its sources to ``${checkouts.dir}/classlib`` directory.
+
+    :``results.dir``:
+        the BTI's directory for storing the test execution results. Usually 
+        test suite uses ``${results.dir}/${suite.name}`` directory for this
+        purpose.
+
+    :``classes.dir``:
+        the directory for compiled java classes. It's needed if the test suite
+        adaptor uses some java classes which should be compiled.
+
+parameters.xml
+--------------
+
+Ant's xml property file containing declarations of parameters describing
+the test suite. This xml property file is loaded after
+``adaptor.xml`` file and, thus, can contain references to the properties 
+defined there. The parameters described in parameters.xml file can be divided 
+into two categories: *execution* and *integration* parameters.
+
+Execution Parameters
+~~~~~~~~~~~~~~~~~~~~
+
+are the parameters affecting the test suite execution. 
+These parameters includes:
+
+*depends*
++++++++++
+
+    contains comma separated list of all of the test suites on which this test
+    suite depends. Actually this parameter defines the order in which the
+    test suite will be executed: it will be executed after suites defined by
+    this parameter. If some of the listed test suites were not selected for
+    execution, they are removed from this parameter. If default value of this
+    parameter does not correspond to selected set of test suites, User *SHOULD*
+    change this parameter by means of ``${suite.name}.parameters.depends``
+    property specified in ``./framework.local.properties`` file.
+
+*shared* parameters 
++++++++++++++++++++
+
+    are the parameters holding the values which can be used by other test
+    suites. Such parameters affecting the execution of other test suites.
+    The path on built product (as JVM for example) can be an example of such
+    parameter.
+
+*required* parameters 
++++++++++++++++++++++
+    
+    are the parameters for which the test suite is not able to provide the
+    default values: the values for these parameters can be specified only
+    from the outside of the test suite. For example: such parameters can
+    hold the path to tested JVM - test suite can't guess this information,
+    so it can't provide the default value for this parameter. On the BTI's
+    ``setup`` stage, all the required parameters for the selected
+    test suites are collected in ``./required-parameters.properties`` file and
+    if some of such parameters has no value, BTI stops with *ERROR* message.
+    
+    **Note:** the declaration of *required* parameter should have a 
+    ``description`` attribute. Without this attribute such a 
+    parameter won't be regarded as *required*.
+    
+    While test suite itself can not provide the default value for such a
+    parameter, it is possible to provide it from BTI's (read "external")
+    side. It is done by means of connecting *required* parameter of test
+    suite with *shared* parameter of test suite which can provide *required*
+    value. Such test suites providing their *shared* parameters should be 
+    listed in ``depends`` parameter of dependent test suite - it will allow
+    them to be executed before dependent test suite, and, correspondingly, 
+    to build a product pointed by the value of *shared* parameter before
+    dependent test suite will use it.
+    The values for *required* parameters can be specified in three places:
+
+    * test suites ``./parameters.xml`` file can contain only connections
+      between *required* parameters and *shared* parameters of some other
+      test suites. These test suites should be listed in ``depends`` parameter.
+
+      **Note:** if these test suites are not selected for execution, these
+      *shared* parameters won't be resolved and *required* parameter won't
+      have the value. BTI will stop with ERROR message requesting to specify
+      the value for this unset *required* parameters.
+
+    * ``./framework.local.properties`` file can contain connections between
+      the *required* and *shared* parameters, as well as direct evaluation of
+      *required* parameters with some value. The value provided in this file
+      takes precedence over the value in ``./parameters.xml`` file.
+
+    * ``./required-parameters.properties`` file can contain only direct
+      evaluation of required parameter with some value. It can't contain any
+      property resolutions. The value provided in this file takes precedence
+      on the values provided in ``./framework.local.properties`` and
+      ``./parameters.xml`` files. 
+      
+      **Note:** the *required* parameters in this file
+      appears in short format: ``${suite.mane}.${required.parameter.name}``. For
+      example, if *required* parameter has been defined in ``parameters.xml`` as
+      ``suite.parameters.required.some.param``, in
+      ``./required-parameters.properties`` it will be in the following form:
+      ``suite.some.param``.
+
+*optional* parameters
++++++++++++++++++++++
+
+      are the parameters allowing to user to influence how the test suite
+      will be executed. Such parameters have some default values and should
+      have the description.
+
+
+Integration Parameters
+~~~~~~~~~~~~~~~~~~~~~~
+
+describe the support from BTI's side needed for correct test suite execution.
+These parameters includes:
+
+*external* parameters
++++++++++++++++++++++
+
+      are the parameters describing the dependencies of the test suite on
+      external resources located at some point of the web. This parameters are
+      addressed to the `External Dependencies Fetching Module`_ which is
+      responsible for downloading and sharing them between the test suites.
+
+*cc* parameters
++++++++++++++++
+
+      are the parameters describing test suite's specific of continuous
+      execution. These parameters describe files and SVN repositories monitored
+      by test suite, define bootstrapping and publishing procedures, do some
+      test suite's specific configurations of `Execution Status Notifier`_
+      module. For more information see `CTE Module`_ section.
+
+*mail* parameters
++++++++++++++++++
+
+      are the parameters describing the test suite's specific configuration of
+      `Execution Status Notifier`_ module. Initially these parameters were
+      *cc* parameters, but now they have been moved into special category.
+      The parameters specified here take precedence over corresponding *cc*
+      ones. This parameters allows to specify the summary information to be
+      sent with mail notifications, files to be attached, and so on. See
+      `Execution Status Notifier`_ for more details.
+
+
+
+CTE Module
+==========
+
+Continuous Test Execution module consists of the following source files::
+
+    ./scripts/cc-project.xml
+    ./scripts/cc-exec-adaptor.xml
+    ./scripts/templates/cc-config-template.xml
+    ./scripts/templates/cc-project-template.xml
+
+This module is responsible for performing of continuous test execution
+launched by ``run-cc`` BTI's target. The work of this module can be divided into
+2 steps: `preparation`_ and `execution`_.
+
+Preparation
+-----------
+
+On this stage, CTE Module gathers the parameters describing CruiseControl-related
+configuration and generates CC project files (CC configurations) for each of
+selected test suites. The parameters defining these configurations are
+provided by two sources:
+
+- BTI defines *global* CTE parameters common for all test suites. Such
+  parameters include the information about web ports, timeouts, etc. and
+  can be (along with their descriptions) seen in ``./scripts/parameters.xml``
+  file under ``<cc>`` section.
+
+  **Note:**
+  These parameters are *user configurable*. It means that their
+  values can be changed by user through the corresponding properties
+  specified in ``./framework.local.properties`` file.
+
+- Each of selected test suites uses its own set of CTE parameters
+  describing the specific of its continuous execution. These parameters
+  appear in test suites' ``parameters.xml`` file under ``<cc>`` section: 
+
+    :``trigger``: - (no value by default)
+      - path to local file which modification will trigger the execution 
+        of test suite.
+
+    :``usesvn``:  - (no value by default)
+      - the URL of SVN repository which modification will trigger the 
+        test suite's execution. There can several URLs be set. 
+        Every other parameter with ``usesvn.`` prefix is considered as URL 
+        for SVN repository.
+
+    :``publish``: - (default is ``true``)
+      - determines whether do status notification publishing for this test
+        suite or not. The value of this parameter can be changed by user
+        through the ``./framework.local.properties`` configuration file.
+        In future this parameter will go to ``<mail>`` section of parameters
+        configuration file (so it won't be a CTE configuration parameter).
+        
+    :``results``: - (no value by default)
+      - path to directory containing the test execution result files in XML
+        JUnit format for status notification message. This parameter will 
+        go to ``<mail>`` section.
+
+  The `Execution Status Notifier`_ configuration parameters may also
+  appear in ``<cc>`` section. But their normal place has been moved to 
+  ``<mail>`` section. They are: ``summary``, ``message``, and ``attachs``.
+  They describe summary, content of the notification message and files to be
+  attached with the message correspondingly.
+
+  **Note:**
+  These parameters (except ``publish``) are `Integration Parameters`_ and
+  are *not user configurable*.  It means that their values should not be changed
+  by user.
+
+Execution
+---------
+
+After ``setup`` of each of the selected test suites, BTI launches CruiseControl
+with generated configuration. CC launches test suites in order described by
+``test.suites.names.list`` property defined in ``./selected-suites.properties``
+file generated on ``setup`` stage. 
+If some of the test suites failed, dependent test suites won't be run.
+
+The execution log of each of the test suites
+can be found in CC's web interface under *build artifacts* section. Web
+interface of CC is available at localhost with
+``framework.parameters.cc.webport`` port number which is 8080 by default.
+
+As stated in `README`_ file, there are two modes of continuous execution: 
+SVN modifications triggered and scheduled runs. SVN modifications triggered runs 
+are executed each time monitored SVN of trigger file modifications occur. 
+Scheduled runs are launched at the particular time specified by the following
+properties in framework.local.properties file:
+
+    ``framework.parameters.schedule.day=Tuesday``
+       Do launch test suites execution on particular day. 
+       If this property is not defined, the test run will be launched everyday.
+
+    ``framework.parameters.schedule.time=1530``
+       Do launch test suites execution at this particular time.
+       Should be defined for scheduled execution mode.
+
+
+
+Execution Status Notifier
+=========================
+
+Implemented by the following source file::
+
+    ./scripts/publisher.xml
+
+This module performs the composition and sending of the message with
+test suite execution status. The work of this module is regulated by means of
+particular configuration parameters defined either by BTI or by
+test suites. The parameters defined by BTI have the global influence and
+take effect for all of the suites. Such the parameters are (look in
+``./scripts/parameters.xml`` file for more info):
+
+    :``framework.parameters.usemail``:
+          the notification publishing will be in use only if this parameter is
+          set and contains the address of SMTP server to be used for mailing.
+
+    :``framework.parameters.usemail.always``:
+          defines the notification mode. If it holds ``true`` value - the
+          notifications will be sent regardless of execution status. 
+          But if it holds ``false`` - notifications will be sent only after 
+          execution status changes (from *PASSED* to *FAILED*, and vice versa)
+
+    :``framework.parameters.usemail.to``:
+          should contain the address to where notifications should be sent.
+          Use comma for multiple values.
+
+    :``framework.parameters.usemail.from``:
+          should contain the address from where notifications should be sent
+          Use comma for multiple values.
+
+    :``framework.parameters.usemail.suppress.exelogs``:
+          controls whether execution logs will be sent with failure messages or
+          not.
+
+    :``framework.parameters.usemail.subject.prefix``:
+          sets up the prefix for notifications subject line.
+
+Also each of the test suite can provide its own parameters configuring the
+status notifications. They are described in test suite's parameters.xml file
+under <mail> section and their values should not be changed by user:
+
+    :``summary``:
+          the path to file holding test suite's execution summary. This content
+          will be prepended to the notification message
+
+    :``message``:
+          the path to file holding test suite's message content. This content
+          will replace one generated by default.
+
+    :``attachs``:
+          comma separated paths to files to be attached with notification.
+
+Sometimes it is necessary for test suite to override some global parameter
+defined by framework's parameters. It can be done by means of parameters
+specified in ``./framework.local.properties`` file having
+``${suite.name}.parameters.mail`` prefix instead of 
+``framework.parameters.usemail`` one.
+
+For example, if the User needs for *test_suite* to use particular destination
+mail address, the following property can do that:
+
+``test_suite.parameters.mail.to=different@address.info``
+
+In this case status notifications for all of the test suites will be sent to 
+the address specified by ``framework.parameters.usemail.to`` parameter,
+but notifications for *test_suite* will be sent to *different@address.info*.
+
+
+
+External Dependencies Fetching Module
+=====================================
+  
+With this module BTI offers the common way to fetch, use and share external
+libraries between the test suites. It is done by means of parameters described
+in ``<external>`` section of ``parameters.xml`` files. The description of the 
+external library should contain at least 2 fields: URL from where to take the 
+dependency, and MD5 value to check the dependency. Also there can be some
+attributes describing the type of the dependency and some installation
+information.
+
+BTI on ``setup`` stage fetches all the dependencies according to their
+descriptions and sets up corresponding properties to access them.
+For example: let's look at the following description::
+
+    <external>
+        <my-dependency
+            url="http://host.of/my-dependency.zip"
+            md5="87A617D7E65979AE01B5ACE4649EB40B"
+            dir="my-dep-dir"
+            inc="*/lib/*.jar"
+        />
+    </external>
+
+It describes dependency called *my-dependency* which is located at specified
+URL and has particular MD5 check sum. During the local installation of
+downloaded archive only jar files from `lib` directory will be unzipped and
+placed under `my-dep-dir` directory. The path to this directory will be
+provided by ``ext.my-dependency.location`` property.
+
+This module is under development and provides only simple functionality.
+For working usage examples, please, look at ``./scripts/parameters.xml`` file.
+
+------------------------------------------------------------------------------
+
+.. [x] Other brands and names are the property of their respective owners
+.. _README: ./README.txt
+
+.. Use "Vim reStructured Text" utility to generate HTML:
+        http://skawina.eu.org/mikolaj/vst.html
+   or online renderer:
+        http://www.hosting4u.cz/jbar/rest/rest.html
+
+.. vim:filetype=rst
+

Propchange: harmony/enhanced/buildtest/branches/2.0/dev-guide.txt
------------------------------------------------------------------------------
    svn:eol-style = native



Mime
View raw message