brooklyn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Bouron <thomas.bou...@cloudsoftcorp.com>
Subject Re: Request for comment on a proposal for Blueprint-based Test Support in Brooklyn
Date Fri, 30 Oct 2015 08:08:10 GMT
+100 as well!

I quite like what the test-case could look like in a future (dont't get me
wrong, it's already great as it is!)
Anyway, that will ease entities' live tests quite a lot.

Best.

On Fri, 30 Oct 2015 at 06:13 Andrea Turli <andrea.turli@cloudsoftcorp.com>
wrote:

> +100 :)
>
> Particularly I like the incremental approach and the clear mid-term
> roadmap.
>
> Let's do it ;)
>
> Andrea
>
> Il giorno gio 29 ott 2015 21:16 Geoff Macartney <
> geoff.macartney@cloudsoftcorp.com> ha scritto:
>
> > Hi Brooklyners,
> >
> > We’ve been thinking on how we can make it easy to write tests for
> entities
> > in policies. There follows a proposal for a new feature in Brooklyn to
> > provide such tests directly in YAML Blueprints. We request comments from
> > the community on the idea, and would very much welcome suggestions for
> > developing it.
> >
> >
> >
> > Description of Proposal
> >
> > To ensure the quality of blueprints in the Brooklyn ecosystem, we propose
> > some additional test support functionality in Brooklyn.  This consists of
> > additional entities which, through composition and configuration, allow
> for
> > thorough coverage of blueprint test scenarios.  In addition, these tests
> > will provide a good illustration of how blueprints are intended be used.
> >
> > For ease of use, we are looking to support the specification of test
> cases
> > as YAML blueprints which refer to the system under test and define one or
> > more test scenarios.  These tests could then be invoked manually, simply
> by
> > deploying them, or by the maven plugin or an automated test framework.
> >
> > Over time, this test functionality should include:
> >
> > - Ability to operate directly upon the entities in Blueprints, testing
> > sensor values, invoking effectors, checking policies, etc.
> > - Support for Rebind, i.e. the ability to restart the Brooklyn server,
> > such that the persisted state of running instances is correctly restored.
> > - Backwards compatibility is also important - persisted state from
> > previous versions should work with future versions of Brooklyn.
> > - Blueprint upgrade: for a running instance of the blueprint, upgrade
> from
> > one version of the blueprint to another should be tested.
> > - Extensibility: A test framework must be extensible: as well as generic
> > assertions (e.g. service reports as up), there needs to be more specific
> > health checks (e.g. port is reachable and returns correct HTTP code).
> >
> > These tests will also help with the QA when releasing a new version of
> > Brooklyn, but their primary purpose is to provide for QA of individual
> > entities.  Non-goal:  these tests are not intended to duplicate or be a
> > substitute for the unit tests and integration tests that should be
> > written/run in Brooklyn and in other downstream projects.
> >
> > The work must be incremental: the first early deliverables should provide
> > immediate value, while allowing more advanced features to be added over
> > time.
> >
> >
> >
> > Draft Implementation
> >
> > To make the ideas above concrete, we have created example code [1].
> >
> > This includes the following new entities:
> >
> > - org.apache.brooklyn.test.framework.TestSensor: Checks the value of a
> > sensor against an expected value.
> > - org.apache.brooklyn.test.framework.TestEffector: Invoke an effector on
> > an entity.
> > - org.apache.brooklyn.test.framework.TestCase: A ‘container’ class for
> > Test entities.  TestCases have brooklyn.children that are started in
> order.
> > - org.apache.brooklyn.test.framework.ParallelTestCase: Same as TestCase
> > but runs its children in parallel.
> > - org.apache.brooklyn.test.framework.TestHttpCall: Do an HTTP GET and
> > check the results.
> >
> >
> >
> > Example Blueprint
> >
> > This is an example of a blueprint that contains a test. It deploys a
> > Tomcat Server, then uses TestSensor to check it is running (sensor
> > service.isUp).  Next it tests that the main URI of the Tomcat front page
> > becomes accessible, with TestHttpCall. It then stops the Tomcat using a
> > TestEffector, and then again invokes the sensor test, this time with an
> > expectation that the service is not running (service.isUp = false).
> >
> >
> > name: Webapp Test
> > location: localhost
> > services:
> > - type: org.apache.brooklyn.test.framework.TestCase
> >   brooklyn.children:
> >   - type: org.apache.brooklyn.entity.webapp.tomcat.TomcatServer
> >     id: tomcat
> >     brooklyn.config:
> >       war: "
> >
> http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war
> > "
> >   - type: org.apache.brooklyn.test.framework.TestSensor
> >     name: tomcat comes up
> >     target: $brooklyn:component("tomcat")
> >     sensor: service.isUp
> >     timeout: 5m
> >     assert:
> >       equals: true
> >   - type: org.apache.brooklyn.test.framework.TestHttpCall
> >     name: tomcat root context accessible
> >     url: url:
> $brooklyn:component("tomcat").attributeWhenReady("main.uri")
> >     assert:
> >       status: 200
> >   - type: org.apache.brooklyn.test.framework.TestEffector
> >     name: stop tomcat
> >     target: $brooklyn:component("tomcat")
> >     effector: stop
> >   - type: org.apache.brooklyn.test.framework.TestSensor
> >     name: tomcat goes down
> >     target: $brooklyn:component("tomcat")
> >     sensor: service.isUp
> >     assert:
> >       equals: false
> >
> >
> > TestCase will invoke its children sequentially.  The types may be added
> to
> > the catalog as `test-case`, `test-sensor`, and `test-effector`; and we
> > might support `target` and `tests` as more descriptive alternatives to
> > `brooklyn.children`, with a `target` entity becoming the default target
> for
> > sub-tests.  Thus the above test case could also in future be written as:
> >
> >
> > name: Webapp Test
> > location: localhost
> > services:
> > - type: test-case
> >   name: Stop Test
> >   target:
> >     type: org.apache.brooklyn.entity.webapp.tomcat.TomcatServer
> >     id: tomcat
> >     brooklyn.config:
> >       war: "
> >
> http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war
> > "
> >   tests:
> >   - type: test-sensor
> >     name: tomcat comes up
> >     sensor: service.isUp
> >     timeout: 5m
> >     assert:
> >       equals: true
> >   - type: test-httpcall
> >     name: tomcat root context accessible
> >     url: url:
> $brooklyn:component("tomcat").attributeWhenReady("main.uri")
> >     assert:
> >       status: 200
> >   - type: test-effector
> >     name: stop tomcat
> >     effector: stop
> >   - type: test-sensor
> >     name: tomcat goes down
> >     sensor: service.isUp
> >     assert:
> >       equals: false
> >
> > Because the individual tests are normal entities, they could be added to
> > the catalog in the normal way, so that for instance new types
> > test-service-is-up and test-service-is-down could easily be defined to
> > replace the first and last tests with a single line.
> >
> > More complex test scenarios, such as launching a load test script, could
> > be achieved simply by writing your own entity to conduct the scenario; it
> > could be pointed at the target, given its own ID, and that entity
> > referenced as part of subsequent test-sensor and test-effector tests.
> >
> > Over time we see the library of test entities growing, including actions
> > like testing HTTP calls and performing rebind into different servers, and
> > we see the library of tests growing, so that tests become a critical part
> > of any new blueprint.
> >
> > References
> >
> > [1]
> https://github.com/m4rkmckenna/incubator-brooklyn/tree/test-framework
> >
> >
> > Chris Burke (christopher.burke@cloudsoft.io)
> > David Lloyd (david.lloyd@cloudsoft.io)
> > John McCabe (john.mccabe@cloudsoft.io)
> > Mark McKenna (mark.mckenna@cloudsoft.io)
> > Geoff Macartney (geoff.macartney@cloudsoft.io)
> >
> >
>
-- 
Thomas Bouron • Software Engineer @ Cloudsoft Corporation •
http://www.cloudsoftcorp.com/
Github: https://github.com/tbouron
Twitter: https://twitter.com/eltibouron

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message