brooklyn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geoff Macartney <geoff.macart...@cloudsoftcorp.com>
Subject Request for comment on a proposal for Blueprint-based Test Support in Brooklyn
Date Thu, 29 Oct 2015 20:16:08 GMT
Hi Brooklyners,

We’ve been thinking on how we can make it easy to write tests for entities in policies.
There follows a proposal for a new feature in Brooklyn to provide such tests directly in YAML
Blueprints. We request comments from the community on the idea, and would very much welcome
suggestions for developing it.



Description of Proposal

To ensure the quality of blueprints in the Brooklyn ecosystem, we propose some additional
test support functionality in Brooklyn.  This consists of additional entities which, through
composition and configuration, allow for thorough coverage of blueprint test scenarios.  In
addition, these tests will provide a good illustration of how blueprints are intended be used.

For ease of use, we are looking to support the specification of test cases as YAML blueprints
which refer to the system under test and define one or more test scenarios.  These tests could
then be invoked manually, simply by deploying them, or by the maven plugin or an automated
test framework.

Over time, this test functionality should include:

- Ability to operate directly upon the entities in Blueprints, testing sensor values, invoking
effectors, checking policies, etc.
- Support for Rebind, i.e. the ability to restart the Brooklyn server, such that the persisted
state of running instances is correctly restored.
- Backwards compatibility is also important - persisted state from previous versions should
work with future versions of Brooklyn.
- Blueprint upgrade: for a running instance of the blueprint, upgrade from one version of
the blueprint to another should be tested.
- Extensibility: A test framework must be extensible: as well as generic assertions (e.g.
service reports as up), there needs to be more specific health checks (e.g. port is reachable
and returns correct HTTP code). 

These tests will also help with the QA when releasing a new version of Brooklyn, but their
primary purpose is to provide for QA of individual entities.  Non-goal:  these tests are not
intended to duplicate or be a substitute for the unit tests and integration tests that should
be written/run in Brooklyn and in other downstream projects.

The work must be incremental: the first early deliverables should provide immediate value,
while allowing more advanced features to be added over time.



Draft Implementation

To make the ideas above concrete, we have created example code [1].

This includes the following new entities:

- org.apache.brooklyn.test.framework.TestSensor: Checks the value of a sensor against an expected
value. 
- org.apache.brooklyn.test.framework.TestEffector: Invoke an effector on an entity.
- org.apache.brooklyn.test.framework.TestCase: A ‘container’ class for Test entities.
 TestCases have brooklyn.children that are started in order.
- org.apache.brooklyn.test.framework.ParallelTestCase: Same as TestCase but runs its children
in parallel.
- org.apache.brooklyn.test.framework.TestHttpCall: Do an HTTP GET and check the results.



Example Blueprint

This is an example of a blueprint that contains a test. It deploys a Tomcat Server, then uses
TestSensor to check it is running (sensor service.isUp).  Next it tests that the main URI
of the Tomcat front page becomes accessible, with TestHttpCall. It then stops the Tomcat using
a TestEffector, and then again invokes the sensor test, this time with an expectation that
the service is not running (service.isUp = false).


name: Webapp Test
location: localhost
services:
- type: org.apache.brooklyn.test.framework.TestCase
  brooklyn.children:
  - type: org.apache.brooklyn.entity.webapp.tomcat.TomcatServer
    id: tomcat
    brooklyn.config:
      war: "http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war"
  - type: org.apache.brooklyn.test.framework.TestSensor
    name: tomcat comes up
    target: $brooklyn:component("tomcat")
    sensor: service.isUp
    timeout: 5m
    assert:
      equals: true
  - type: org.apache.brooklyn.test.framework.TestHttpCall
    name: tomcat root context accessible
    url: url: $brooklyn:component("tomcat").attributeWhenReady("main.uri")
    assert:
      status: 200
  - type: org.apache.brooklyn.test.framework.TestEffector
    name: stop tomcat
    target: $brooklyn:component("tomcat")
    effector: stop
  - type: org.apache.brooklyn.test.framework.TestSensor
    name: tomcat goes down
    target: $brooklyn:component("tomcat")
    sensor: service.isUp
    assert:
      equals: false


TestCase will invoke its children sequentially.  The types may be added to the catalog as
`test-case`, `test-sensor`, and `test-effector`; and we might support `target` and `tests`
as more descriptive alternatives to `brooklyn.children`, with a `target` entity becoming the
default target for sub-tests.  Thus the above test case could also in future be written as:


name: Webapp Test
location: localhost
services:
- type: test-case
  name: Stop Test
  target:
    type: org.apache.brooklyn.entity.webapp.tomcat.TomcatServer
    id: tomcat
    brooklyn.config:
      war: "http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war"
  tests:
  - type: test-sensor
    name: tomcat comes up
    sensor: service.isUp
    timeout: 5m
    assert:
      equals: true
  - type: test-httpcall
    name: tomcat root context accessible
    url: url: $brooklyn:component("tomcat").attributeWhenReady("main.uri")
    assert:
      status: 200
  - type: test-effector
    name: stop tomcat
    effector: stop
  - type: test-sensor
    name: tomcat goes down
    sensor: service.isUp
    assert:
      equals: false

Because the individual tests are normal entities, they could be added to the catalog in the
normal way, so that for instance new types test-service-is-up and test-service-is-down could
easily be defined to replace the first and last tests with a single line.

More complex test scenarios, such as launching a load test script, could be achieved simply
by writing your own entity to conduct the scenario; it could be pointed at the target, given
its own ID, and that entity referenced as part of subsequent test-sensor and test-effector
tests.

Over time we see the library of test entities growing, including actions like testing HTTP
calls and performing rebind into different servers, and we see the library of tests growing,
so that tests become a critical part of any new blueprint.

References

[1] https://github.com/m4rkmckenna/incubator-brooklyn/tree/test-framework


Chris Burke (christopher.burke@cloudsoft.io)
David Lloyd (david.lloyd@cloudsoft.io)
John McCabe (john.mccabe@cloudsoft.io)
Mark McKenna (mark.mckenna@cloudsoft.io)
Geoff Macartney (geoff.macartney@cloudsoft.io)


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message