Mailing-List: contact commits-help@brooklyn.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@brooklyn.apache.org
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
From: drigodwin@apache.org
To: commits@brooklyn.apache.org
Date: Fri, 16 Jun 2017 15:21:49 -0000
Message-Id: <c06ccda0af0c4b6ba77f0d7280ae17b4@git.apache.org>
In-Reply-To: <ded7266c75aa4a32872f2346b7a7c24e@git.apache.org>
References: <ded7266c75aa4a32872f2346b7a7c24e@git.apache.org>
Subject: [09/28] brooklyn-docs git commit: Refactor Blueprinting
archived-at: Fri, 16 Jun 2017 15:21:48 -0000

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/multiple-services.md
----------------------------------------------------------------------
diff --git a/guide/blueprints/multiple-services.md b/guide/blueprints/multiple-services.md
new file mode 100644
index 0000000..81ddf07
--- /dev/null
+++ b/guide/blueprints/multiple-services.md
@@ -0,0 +1,97 @@
+---
+title: Multiple Services and Dependency Injection
+layout: website-normal
+---
+
+We've seen the configuration of machines and how to build up clusters.
+Now let's return to our app-server example and explore how more interesting
+services can be configured, composed, and combined.
+
+
+### Service Configuration
+
+We'll begin by using more key-value pairs to configure the JBoss server to run a real app:
+
+{% highlight yaml %}
+{% readj example_yaml/appserver-configured.yaml %}
+{% endhighlight %}
+
+(As before, you'll need to add the `location` info; `localhost` will work for these and subsequent examples.)
+
+When this is deployed, you can see management information in the Brooklyn Web Console,
+including a link to the deployed application (downloaded to the target machine from the `hello-world` URL),
+running on port 8080.
+
+**Tip**:  If port 8080 might be in use, you can specify `8080+` to take the first available port >= 8080;
+the actual port will be reported as a sensor by Brooklyn.
+
+It's also worth indicating an alternate, more formal syntax.
+Not all configuration on entities is supported at the top level of the service specification
+(only those which are defined as "flags" in the underlying blueprint,
+e.g. the `@SetFromFlag("war")` in the `WebAppServiceConstants` parent of `JBoss7Server`).
+All configuration has a formal qualified name, and this can be supplied even where flags or config keys are not
+explicitly defined, by placing it into a `brooklyn.config` section:
+
+{% highlight yaml %}
+{% readj example_yaml/appserver-configured-in-config.yaml %}
+{% endhighlight %}
+
+
+### Multiple Services
+
+If you explored the `hello-world-sql` application we just deployed, 
+you'll have noticed it tries to access a database.
+And it fails, because we have not set one up.  Let's do that now:
+
+{% highlight yaml %}
+{% readj example_yaml/appserver-w-db.yaml %}
+{% endhighlight %}
+
+Here there are a few things going on:
+
+* We've added a second service, which will be the database;
+  you'll note the database has been configured to run a custom setup script
+* We've injected the URL of the second service into the appserver as a Java system property
+  (so our app knows where to find the database) 
+
+**Caution: Be careful if you write your YAML in an editor which attempts to put "smart-quotes" in.
+All quote characters must be plain ASCII, not fancy left-double-quotes and right-double-quotes!**
+
+There are as many ways to do dependency injection as there are developers,
+it sometimes seems; our aim in Brooklyn is not to say this has to be done one way,
+but to support the various mechanisms people might need, for whatever reasons.
+(We each have our opinions about what works well, of course;
+the one thing we do want to call out is that being able to dynamically update
+the injection is useful in a modern agile application -- so we are definitively **not**
+recommending this Java system property approach ... but it is an easy one to demo!)
+
+The way the dependency injection works is again by using the `$brooklyn:` DSL,
+this time referring to the `component("db")` (looked up by the `id` field on our DB component),
+and then to a sensor emitted by that component.
+All the database entities emit a `database.url` sensor when they are up and running;
+the `attributeWhenReady` DSL method will store a pointer to that sensor (a Java Future under the covers)
+in the Java system properties map which the JBoss entity reads at launch time, blocking if needed.
+
+This means that the deployment occurs in parallel, and if the database comes up first,
+there is no blocking; but if the JBoss entity completes its installation and 
+downloading the WAR, it will wait for the database before it launches.
+At that point the URL is injected, first passing it through `formatString`
+to include the credentials for the database (which are defined in the database creation script).
+
+
+### An Aside: Substitutability
+
+Don't like JBoss?  Is there something about Maria?
+One of the modular principles we follow in Brooklyn is substitutability:
+in many cases, the config keys, sensors, and effectors are defined
+in superclasses and are portable across multiple implementations.
+
+Here's an example deploying the same application but with different flavors of the components:
+
+{% highlight yaml %}
+{% readj example_yaml/appserver-w-db-other-flavor.yaml %}
+{% endhighlight %}
+
+We've also brought in the `provisioning.properties` from the VM example earlier
+so our database has 8GB RAM.
+Any of those properties, including `imageId` and `user`, can be defined on a per-entity basis.

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/policies.md
----------------------------------------------------------------------
diff --git a/guide/blueprints/policies.md b/guide/blueprints/policies.md
new file mode 100644
index 0000000..e5f7dde
--- /dev/null
+++ b/guide/blueprints/policies.md
@@ -0,0 +1,198 @@
+---
+title: Policies
+layout: website-normal
+---
+
+Policies perform the active management enabled by Brooklyn.
+They can subscribe to entity sensors and be triggered by them (or they can run periodically,
+or be triggered by external systems).
+
+<!---
+TODO, clarify below, members of what?
+-->
+Policies can add subscriptions to sensors on any entity. Normally a policy will subscribe to its
+associated entity, to the child entities, and/or to the members of a "group" entity.
+
+Common uses of a policy include the following:
+
+*	perform calculations,
+*	look up other values,
+*	invoke effectors  (management policies) or,
+*	cause the entity associated with the policy to emit sensor values (enricher policies).
+
+Entities can have zero or more ``Policy`` instances attached to them.
+
+
+Off-the-Shelf Policies
+----------------------
+
+Policies are highly reusable as their inputs, thresholds and targets are customizable.
+
+### Management Policies
+
+#### AutoScaler Policy
+
+- org.apache.brooklyn.policy.autoscaling.AutoScalerPolicy
+
+Increases or decreases the size of a Resizable entity based on an aggregate sensor value, the current size of the entity, and customized high/low watermarks.
+
+An AutoScaler policy can take any sensor as a metric, have its watermarks tuned live, and target any resizable entity - be it an application server managing how many instances it handles, or a tier managing global capacity.
+
+e.g. if the average request per second across a cluster of Tomcat servers goes over the high watermark, it will resize the cluster to bring the average back to within the watermarks.
+
+{% highlight yaml %}
+brooklyn.policies:
+- type: org.apache.brooklyn.policy.autoscaling.AutoScalerPolicy
+  brooklyn.config:
+    metric: webapp.reqs.perSec.perNode
+    metricUpperBound: 3
+    metricLowerBound: 1
+    resizeUpStabilizationDelay: 2s
+    resizeDownStabilizationDelay: 1m
+    maxPoolSize: 3
+
+{% endhighlight %}
+
+#### ServiceRestarter
+
+- org.apache.brooklyn.policy.ha.ServiceRestarter
+
+Attaches to a SoftwareProcess (or anything Startable, emitting ENTITY_FAILED or other configurable sensor), and invokes restart on failure; if there is a subsequent failure within a configurable time interval, or if the restart fails, this gives up and emits {@link #ENTITY_RESTART_FAILED}
+
+{% highlight yaml %}
+brooklyn.policies:
+- type: org.apache.brooklyn.policy.ha.ServiceRestarter
+  brooklyn.config:
+    failOnRecurringFailuresInThisDuration: 5m
+{% endhighlight %}
+
+#### StopAfterDuration Policy
+
+- org.apache.brooklyn.policy.action.StopAfterDurationPolicy
+
+The StopAfterDurationPolicy can be used to limit the lifetime of an entity.  After a configure time period expires the entity will be stopped.
+
+#### CreateUser Policy
+
+- org.apache.brooklyn.policy.jclouds.os.CreateUserPolicy
+
+The CreateUserPolicy Attaches to an Entity and monitors for the addition of a location to that entity, the policy then adds a new user to the VM with a randomly generated password, with the SSH connection details set on the entity as the createuser.vm.user.credentials sensor.
+
+#### AdvertiseWinRMLogin Policy
+
+- org.apache.brooklyn.location.winrm.WinRmMachineLocation
+
+This is similar to the CreateUserPolicy.  It will monitor the addition of WinRmMachineLocation to an entity and then create a sensor advertising the administrative user's credentials.
+
+#### SshMachineFailureDetector
+
+- org.apache.brooklyn.policy.ha.SshMachineFailureDetector
+
+The SshMachineFailureDetector is an HA policy for monitoring an SshMachine, emitting an event if the connection is lost/restored.
+
+#### ConnectionFailureDetector
+
+- org.apache.brooklyn.policy.ha.ConnectionFailureDetector
+
+The ConnectionFailureDetector is an HA policy for monitoring an http connection, emitting an event if the connection is lost/restored.
+
+#### ServiceReplacer
+
+- org.apache.brooklyn.policy.ha.ServiceReplacer
+
+The ServiceReplacer attaches to a DynamicCluster and replaces a failed member in response to HASensors.ENTITY_FAILED or other sensor.  The [introduction to policies](../) shows a worked example of the ServiceReplacer policy in user.
+
+#### FollowTheSun Policy
+
+- org.apache.brooklyn.policy.followthesun.FollowTheSunPolicy
+
+The FollowTheSunPolicy is for moving work around to follow the demand.  The work can be any Movable entity.  This currently available in yaml blueprints.
+
+#### ConditionalSuspend Policy
+
+- org.apache.brooklyn.policy.ha.ConditionalSuspendPolicy
+
+The ConditionalSuspendPolicy will suspend and resume a target policy based on configured suspend and resume sensors.
+
+#### LoadBalancing Policy
+
+- org.apache.brooklyn.policy.loadbalancing.LoadBalancingPolicy
+
+The LoadBalancingPolicy is attached to a pool of "containers", each of which can host one or more migratable "items".  The policy monitors the workrates of the items and effects migrations in an attempt to ensure that the containers are all sufficiently utilized without any of them being overloaded.
+
+
+Writing a Policy
+----------------
+
+### Your First Policy
+
+Policies perform the active management enabled by Brooklyn.
+Each policy instance is associated with an entity,
+and at runtime it will typically subscribe to sensors on that entity or children,
+performing some computation and optionally actions when a subscribed sensor event occurs.
+This action might be invoking an effector or emitting a new sensor,
+depending the desired behavior is.
+
+Writing a policy is straightforward.
+Simply extend [``AbstractPolicy``](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/core/policy/AbstractPolicy.html),
+overriding the [``setEntity``](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/core/objs/AbstractEntityAdjunct.html#setEntity-org.apache.brooklyn.api.entity.EntityLocal-) method to supply any subscriptions desired:
+
+{% highlight java %}
+    @Override
+    public void setEntity(EntityLocal entity) {
+        super.setEntity(entity)
+        subscribe(entity, TARGET_SENSOR, this)
+    }
+{% endhighlight %}
+
+and supply the computation and/or activity desired whenever that event occurs:
+
+{% highlight java %}
+    @Override
+    public void onEvent(SensorEvent<Integer> event) {
+        int val = event.getValue()
+        if (val % 2 == 1)
+            entity.sayYoureOdd();
+    }
+{% endhighlight %}
+
+
+You'll want to do more complicated things, no doubt,
+like access other entities, perform multiple subscriptions,
+and emit other sensors -- and you can.
+See the best practices below and source code for some commonly used policies and enrichers,
+such as ``AutoScalerPolicy`` and ``RollingMeanEnricher``.
+
+One rule of thumb, to close on:
+try to keep policies simple, and compose them together at runtime;
+for instance, if a complex computation triggers an action,
+define one **enricher** policy to aggregate other sensors and emit a new sensor,
+then write a second policy to perform that action.
+
+
+### Best Practice
+
+The following recommendations should be considered when designing policies:
+
+#### Management should take place as "low" as possible in the hierarchy
+*   place management responsibility in policies at the entity, as much as possible ideally management should take run as a policy on the relevant entity
+
+*   place escalated management responsibility at the parent entity. Where this is impractical, perhaps because two aspects of an entity are best handled in two different places, ensure that the separation of responsibilities is documented and there is a group membership relationship between secondary/aspect managers.
+
+
+#### Policies should be small and composable
+
+e.g. one policy which takes a sensor and emits a different, enriched sensor, and a second policy which responds to the enriched sensor of the first     (e.g. a policy detects a process is maxed out and emits a TOO_HOT sensor; a second policy responds to this by scaling up the VM where it is running, requesting more CPU)
+
+#### Where a policy cannot resolve a situation at an entity, the issue should be escalated to a manager with a compatible policy.
+
+Typically escalation will go to the entity parent, and then cascade up.
+e.g. if the earlier VM CPU cannot be increased, the TOO_HOT event may go to the parent, a cluster entity, which attempts to balance. If the cluster cannot balance, then to another policy which attempts to scale out the cluster, and should the cluster be unable to scale, to a third policy which emits TOO_HOT for the cluster.
+
+#### Management escalation should be carefully designed so that policies are not incompatible
+
+Order policies carefully, and mark sensors as "handled" (or potentially "swallow" them locally), so that subsequent policies and parent entities do not take superfluous (or contradictory) corrective action.
+
+### Implementation Classes
+
+Extend [`AbstractPolicy`](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/core/policy/AbstractPolicy.html), or override an existing policy.

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/salt/about-salt.md
----------------------------------------------------------------------
diff --git a/guide/blueprints/salt/about-salt.md b/guide/blueprints/salt/about-salt.md
new file mode 100644
index 0000000..78f4d47
--- /dev/null
+++ b/guide/blueprints/salt/about-salt.md
@@ -0,0 +1,38 @@
+---
+title: About Salt
+title_in_menu: About Salt
+layout: website-normal
+---
+
+## What you need to know about Salt
+
+Salt is designed to work in either what it calls a 'master/minion' or a 'masterless' topology.
+ 
+In the former, the master server acts as a managing controller for any number of client nodes, called 'minions'. 
+A salt daemon running on the minion connects back to the master server for its operation, and manages the software on 
+the minion according to a specification of 'states' defined in Salt configuration files.  In the latter, there is no 
+master, and the salt daemon on the minion operates based on the Salt files on the minion node.  This is the mode 
+currently supported by Brooklyn.
+
+A 'State' in Salt is a specification of a configuration of some aspect of a system, such as what packages are installed,
+what system services are running, what files exist, etc.  Such states are described in "SLS" files (for SaLt State 
+file). These files are typically written as templates using the "Jinja" templating engine.  The actual SLS files for the
+minion are then created by processing the templates using configuration data provided via Salt's "Pillar" system.
+
+Salt comes with built-in support for many software systems, and has a repository of pre-written Salt states known as 
+'formulas' on GitHub, at [https://github.com/saltstack-formulas](https://github.com/saltstack-formulas).
+
+### How Brooklyn interacts with Salt
+
+Brooklyn provides a Salt entity type. An entity of this type can be specified in a blueprint in order to provision the 
+node through Salt. The Salt entity will download Salt and install it on the node. The entity type supports the 
+configuration of Salt formulas and Pillar data to download, and the configuration of what Salt states to apply. 
+These are managed using Salt in 'masterless' mode, as described on the
+[Saltstack site](https://docs.saltstack.com/en/latest/topics/tutorials/quickstart.html#salt-masterless-quickstart),
+using the 'salt-call' functionality of Salt.
+
+The Salt 'highstate' (the collection of states applied to the system) is exposed as a sensor on the entity.  An effector
+ is provided on the entity that supports general purpose Salt instructions.
+
+
+

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/salt/creating-salt-blueprints.md
----------------------------------------------------------------------
diff --git a/guide/blueprints/salt/creating-salt-blueprints.md b/guide/blueprints/salt/creating-salt-blueprints.md
new file mode 100644
index 0000000..f00d687
--- /dev/null
+++ b/guide/blueprints/salt/creating-salt-blueprints.md
@@ -0,0 +1,163 @@
+---
+title: Creating Blueprints with Salt
+title_in_menu: Creating Blueprints with Salt
+layout: website-normal
+---
+
+To write a blueprint to use Salt with Brooklyn it will help to have a degree of familiarity with Salt itself. In the 
+sections below, when the Brooklyn configuration is described, the underlying Salt operation is also noted briefly, for 
+clarity for readers who know Salt.
+
+To manage a node with Salt, create a blueprint containing a service of type `org.apache.brooklyn.entity.cm.salt.SaltEntity`
+and define the `formulas` and `start_states` 
+For example:
+
+    name: Salt Example setting up Apache httpd
+    location: my-cloud
+    services:
+    - id: httpd-from-salt
+      type: org.apache.brooklyn.entity.cm.salt.SaltEntity
+      formulas:
+      - https://github.com/saltstack-formulas/apache-formula/archive/master.tar.gz
+      start_states:
+      - apache
+    
+This example specifies that Brooklyn should use Salt to download the `apache-formula` from the Saltstack repository on
+Github. The apache formula contains the Apache web server with a simple "it worked" style index page. To start the 
+entity, Brooklyn will use Salt to apply the `apache` state, which will bring up the web server.
+
+A typical usage of the Salt entity might be to include a formula from the Saltstack repository, such as `apache` above,
+and another formula created by the blueprint author, with additional states, such as web site content for the apache 
+server.
+
+### Start States
+
+The `start_states` configuration key defines the top level list of states that will be applied using Salt.  These values
+are added to the Salt `top.sls` file and applied using `state.apply`.  This configuration key is mandatory.
+
+### Stop States
+
+The `stop_states` configuration key is used to specify states that should be applied when the 'stop' effector
+is invoked on the entity.  For example, the Saltstack `mysql` [formula](https://github.com/saltstack-formulas/mysql-formula)
+supplies a state `mysql.disabled` that will shut down the database server.
+
+If the Saltstack formula does not supply a suitable stop state, the blueprint author can create a suitable state and
+include it in an additional formula to be supplied in the `formulas` section. 
+
+The `stop_states` configuration key is optional; 
+if it is not provided, Brooklyn assumes that each state `S` in the `start_states` will have a matching `S.stop` state.  
+If any `S` does not have such a state, the stop effector will fail stopping processes.
+Note that on a machine created for this entity, Brooklyn's default behaviour may be to proceed to destroy the VM,
+so stop states are not always needed unless there is a cleaner shutdown process or you are using long-running servers.
+
+
+### Restart States
+
+For completeness, Brooklyn also provides a `restart_states` configuration key. These states are applied by the restart
+effector, and blueprint authors may choose to provide custom states to implement restart if that is applicable for their
+application. 
+
+This key is again optional.
+If not supplied, Brooklyn will go through each of the states `S` in `start_states` 
+looking for a matching `S.restart` state defined in the formulas.
+If all exist, these will be applied by the restart effector. 
+If none exist, Brooklyn will invoke the `stop` and then `start` effectors -- 
+so `restart` states are not required for Brooklyn to use Salt.
+(If some but not all have matching `restart` states, 
+Brooklyn will fail the restart, on the assumption that the configuration is incomplete.)   
+
+### Formulas
+
+The `formulas` key provides the URLs for archives containing the Salt formulas that defined the required states. These
+archives are downloaded to the `/srv/formula` directory on the minion and added to the state filesystem roots 
+configuration in Salt's minion config, so that their states are available for a `state.apply`.
+
+### Pillar Configuration
+
+A typical Salt deployment will include both states (provided via Salt formulas) and configuration data, provided through 
+Salt's "Pillar" component.  Brooklyn provides configuration keys for the Salt entity to specify where to get the Pillar
+configuration data.  For example:
+
+    name: Salt Example setting up MySQL with a Pillar
+    location: my-cloud
+    services:
+    - id: mysql-from-salt-with-my-pillar
+      type: org.apache.brooklyn.entity.cm.salt.SaltEntity
+    
+      formulas:
+      - https://github.com/saltstack-formulas/mysql-formula/archive/master.tar.gz
+      - http://myhost:8080/my-mysql-formula.tar.gz
+    
+      start_states:
+      - mysql
+      stop_states: 
+      - mysql.disabled
+    
+      pillars: 
+      - mysql
+      pillarUrls:
+      - http://myhost:8080/my-mysql-pillar.tar.gz
+
+
+This blueprint contains the MySQL database, and includes a formula available from `myhost` which includes the schema
+information for the DB. The MySQL formula from Saltstack has extensive configurability through Salt Pillars. In the 
+blueprint above, Brooklyn is instructed to apply the pillars defined in the `pillars` configuration key.  (This will 
+add these values to the Salt Pillars `top.sls` file.)  The pillar data must be downloaded; for this, the `pillarUrls` key
+provides the address of an archive containing the Pillar data.  The contents of the archive will be extracted and put
+in the `/srv/pillar` directory on the minion, in order to be available to Salt when applying the pillar. For example,
+the archive above can simply have the structure
+
+    pillar/
+    |
+    +- mysql/
+       |
+       +- init.sls
+
+The init.sls contains the pillar configuration values, such as 
+
+    # Manage databases
+    database:
+      - orders
+    schema:
+      orders:
+        load: True
+        source: salt://mysql/files/orders.schema
+
+Meanwhile the `my-mysql-formula.tar.gz` formula archive contains the schema:
+
+    my-mysql-formula/
+    |
+    +- mysql/
+       |
+       +- files/
+          |
+          +- orders.schema
+
+Note that the blueprint above defines an `id` for the Salt entity.  This id, if provided, is set as the minion id in
+the Salt configuration file.  This is useful particularly for Pillar configuration, as, if there are more than one 
+Salt managed nodes in the application, they can share a common Pillar file, with appropriate subdivisions of pillar 
+data based on minion id.
+
+### Highstate Sensors
+
+The Salt entity exposes the Salt "highstate" on the node via Brooklyn sensors.  Firstly a single sensor `salt.states` 
+contains a list of all the top level Salt state ID declarations in the highstate.  For example, for the mysql case 
+above, this might look like:
+
+    ["mysql_additional_config", "mysql_config", "mysql_db_0", "mysql_db_0_load", "mysql_db_0_schema", "mysql_debconf",
+     "mysql_debconf_utils", "mysql_python", "mysql_user_frank_localhost", "mysql_user_frank_localhost_0", 
+     "mysql_user_nopassuser_localhost", "mysqld"]
+
+Then, for each ID and each Salt state function in that ID, a Brooklyn sensor is created, containing a map of the data
+from the highstate.  For example, the `salt.state.mysqld.service.running` sensor would have a value like:
+
+    {"name":"mysql", "enable":true, "watch":[{"pkg":"mysqld"}, {"file":"mysql_config"}], "order":10005}
+
+
+### saltCall Effector
+
+The Salt entity includes a general purpose Salt effector, `saltCall`, which permits execution of Salt commands via
+`salt-call --local`.  It contains a single parameter, `spec`, which specifies the command to invoke.  For example, 
+invoking the effector with a `spec` value of `network.interfaces --out=yaml` would return a YAML formatted map of the 
+network interfaces on the minion.
+

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/salt/index.md
----------------------------------------------------------------------
diff --git a/guide/blueprints/salt/index.md b/guide/blueprints/salt/index.md
new file mode 100644
index 0000000..265e07f
--- /dev/null
+++ b/guide/blueprints/salt/index.md
@@ -0,0 +1,16 @@
+---
+title: Salt in YAML Blueprints
+layout: website-normal
+children:
+- about-salt.md
+- creating-salt-blueprints.md
+---
+
+This guide describes how Brooklyn entities can be created using the Salt infrastructure management tool
+ ([saltstack.com](https://saltstack.com/)).
+At present Brooklyn provides basic support for Salt, operating in a 'masterless' mode. 
+Comments on this support and suggestions for further development are welcome.
+
+This guide assumes you are familiar with the basics of [creating YAML blueprints](../).
+
+{% include list-children.html %}

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/setting-locations.md
----------------------------------------------------------------------
diff --git a/guide/blueprints/setting-locations.md b/guide/blueprints/setting-locations.md
new file mode 100644
index 0000000..91efc5f
--- /dev/null
+++ b/guide/blueprints/setting-locations.md
@@ -0,0 +1,132 @@
+---
+title: Setting Locations
+layout: website-normal
+toc: ../guide_toc.json
+categories: [use, guide, defining-applications]
+---
+
+{% include fields.md %}
+
+Brooklyn supports a very wide range of target locations. 
+With deep integration to [Apache jclouds](https://jclouds.apache.org), most well-known clouds 
+and cloud platforms are supported. See the [Locations guide]({{ site.path.guide }}/locations/) 
+for details and more examples.
+
+### Cloud Example
+
+The following example is for Amazon EC2:
+
+{% highlight yaml %}
+{% readj example_yaml/simple-appserver-with-location.yaml %}
+{% endhighlight %}
+
+(You'll need to replace the `identity` and `credential` with the 
+"Access Key ID" and "Secret Access Key" for your account,
+as configured in the [AWS Console](https://console.aws.amazon.com/iam/home?#security_credential).)
+
+Other popular public clouds include `softlayer`, `google-compute-engine`, and `rackspace-cloudservers-us`.
+Private cloud systems including `openstack-nova` and `cloudstack` are also supported,
+although for these you'll supply an `endpoint: https://9.9.9.9:9999/v2.0/` 
+(or `client/api/` in the case of CloudStack) instead of the `region`.
+
+
+### "Bring Your Own Nodes" (BYON) Example
+
+You can also specify pre-existing servers to use -- "bring-your-own-nodes".
+The example below shows a pool of machines that will be used by the entities within the 
+application.
+
+{% highlight yaml %}
+{% readj example_yaml/simple-appserver-with-location-byon.yaml %}
+{% endhighlight %}
+
+
+### Single Line and Multi Line Locations
+
+A simple location can be specified on a single line. Alternatively, it can be split to have one
+configuration option per line (recommended for all but the simplest locations).
+
+For example, the two examples below are equivalent:
+
+{% highlight yaml %}
+location: byon(name="my loc",hosts="1.2.3.4",user="bob",privateKeyFile="~/.ssh/bob_id_rsa")
+{% endhighlight %}
+
+{% highlight yaml %}
+location:
+  byon:
+    name: "my loc"
+    hosts:
+    - "1.2.3.4"
+    user: "bob"
+    privateKeyFile: "~/.ssh/bob_id_rsa"
+{% endhighlight %}
+
+
+### Specific Locations for Specific Entities
+
+One can define specific locations on specific entities within the blueprint (instead of, or as 
+well as, defining the location at the top-level of the blueprint).
+
+The example below will deploy Tomcat and JBoss App Server to different Bring Your Own Nodes
+locations:
+
+{% highlight yaml %}
+{% readj example_yaml/simple-appserver-with-location-per-entity.yaml %}
+{% endhighlight %}
+
+The rules for precedence when defining a location for an entity are:
+
+* The location defined on that specific entity.
+* If no location is defined, then the first ancestor that defines an explicit location.
+* If still no location is defined, then the location defined at the top-level of the blueprint.
+
+This means, for example, that if you define an explicit location on a cluster then it will be used 
+for all members of that cluster.
+
+
+### Multiple Locations
+
+Some entities are written to expect a set of locations. For example, a `DynamicFabric` will
+create a member entity in each location that it is given. To supply multiple locations, simply
+use `locations` with a yaml list.
+
+In the example below, it will create a cluster of app-servers in each location. One location is
+used for each `DynamicCluster`; all app-servers inside that cluster will obtain a machine from
+that given location.
+
+{% highlight yaml %}
+{% readj example_yaml/fabric-with-multiple-locations.yaml %}
+{% endhighlight %}
+
+The entity hierarchy at runtime will have a `DynamicFabric` with two children, each of type 
+`DynamicCluster` (each running in different locations), each of which initially has three 
+app-servers.
+ 
+For brevity, this example excludes the credentials for aws-ec2. These could either be specificed
+in-line or defined as named locations in the catalog (see below).
+
+
+### Adding Locations to the Catalog
+
+The examples above have given all the location details within the application blueprint.
+It is also possible (and indeed preferred) to add the location definitions to the catalog
+so that they can be referenced by name in any blueprint.
+
+For more information see the [Operations: Catalog]({{ site.path.guide }}/blueprints/catalog/) section of 
+the User Guide.
+
+
+### Externalized Configuration
+
+For simplicity, the examples above have included the cloud credentials. For a production system, 
+it is strongly recommended to use [Externalized Configuration]({{ site.path.guide }}/ops/externalized-configuration.html)
+to retrieve the credentials from a secure credentials store, such as [Vault](https://www.vaultproject.io).
+
+
+### Use of provisioning.properties
+
+An entity that represents a "software process" can use the configuration option 
+`provisioning.properties` to augment the location's configuration. For more information, see
+[Entity Configuration]({{ site.path.guide }}/blueprints/entity-configuration.html#entity-provisioningproperties-overriding-and-merging)
+details.

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/example_yaml/entities/infrastructuredeploymenttestcase-entity.yaml
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/example_yaml/entities/infrastructuredeploymenttestcase-entity.yaml b/guide/blueprints/test/example_yaml/entities/infrastructuredeploymenttestcase-entity.yaml
new file mode 100644
index 0000000..6b344da
--- /dev/null
+++ b/guide/blueprints/test/example_yaml/entities/infrastructuredeploymenttestcase-entity.yaml
@@ -0,0 +1,11 @@
+- type: org.apache.brooklyn.test.framework.InfrastructureDeploymentTestCase
+  brooklyn.config:
+    infrastructure.deployment.location.sensor: entity.dynamicLocation
+    infrastructure.deployment.spec:
+      $brooklyn:entitySpec:
+        - type: docker-cloud-calico
+          ...
+    infrastructure.deployment.entity.specs:
+      - $brooklyn:entitySpec:
+          type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess
+          ...
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/example_yaml/entities/loopovergroupmembers-entity.yaml
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/example_yaml/entities/loopovergroupmembers-entity.yaml b/guide/blueprints/test/example_yaml/entities/loopovergroupmembers-entity.yaml
new file mode 100644
index 0000000..713ffbc
--- /dev/null
+++ b/guide/blueprints/test/example_yaml/entities/loopovergroupmembers-entity.yaml
@@ -0,0 +1,6 @@
+- type: org.apache.brooklyn.test.framework.LoopOverGroupMembersTestCase
+  target: $brooklyn:entity("infrastructure").component("child", "DockerHosts")
+  testSpec:
+    $brooklyn:entitySpec:
+      type: org.apache.brooklyn.test.framework.TestSensor
+      ...
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/example_yaml/entities/paralleltestcase-entity.yaml
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/example_yaml/entities/paralleltestcase-entity.yaml b/guide/blueprints/test/example_yaml/entities/paralleltestcase-entity.yaml
new file mode 100644
index 0000000..ccd0c0b
--- /dev/null
+++ b/guide/blueprints/test/example_yaml/entities/paralleltestcase-entity.yaml
@@ -0,0 +1,6 @@
+- type: org.apache.brooklyn.test.framework.ParallelTestCase
+  brooklyn.children:
+  - type: org.apache.brooklyn.entity.database.mysql.MySqlNode
+    ...
+  - type: org.apache.brooklyn.entity.webapp.ControlledDynamicWebAppCluster
+    ...
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/example_yaml/entities/script1.sh
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/example_yaml/entities/script1.sh b/guide/blueprints/test/example_yaml/entities/script1.sh
new file mode 100644
index 0000000..2a98304
--- /dev/null
+++ b/guide/blueprints/test/example_yaml/entities/script1.sh
@@ -0,0 +1,2 @@
+#!/bin/sh
+echo hello world
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/example_yaml/entities/testcase-entity.yaml
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/example_yaml/entities/testcase-entity.yaml b/guide/blueprints/test/example_yaml/entities/testcase-entity.yaml
new file mode 100644
index 0000000..0e1aa00
--- /dev/null
+++ b/guide/blueprints/test/example_yaml/entities/testcase-entity.yaml
@@ -0,0 +1,6 @@
+- type: org.apache.brooklyn.test.framework.TestCase
+  brooklyn.children:
+  - type: org.apache.brooklyn.entity.database.mysql.MySqlNode
+    ...
+  - type: org.apache.brooklyn.test.framework.TestSensor
+    ...
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/example_yaml/entities/testeffector-entity.yaml
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/example_yaml/entities/testeffector-entity.yaml b/guide/blueprints/test/example_yaml/entities/testeffector-entity.yaml
new file mode 100644
index 0000000..e9d570e
--- /dev/null
+++ b/guide/blueprints/test/example_yaml/entities/testeffector-entity.yaml
@@ -0,0 +1,8 @@
+- type: org.apache.brooklyn.test.framework.TestEffector
+  name: Invoke Deploy Effector
+  target: $brooklyn:entity("tomcat")
+  effector: deploy
+  timeout: 5m
+  params:
+    url: http://search.maven.org/remotecontent?filepath=org/apache/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.8.0-incubating/brooklyn-example-hello-world-sql-webapp-0.8.0-incubating.war
+    targetName: newcontext
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/example_yaml/entities/testhttpcall-entity.yaml
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/example_yaml/entities/testhttpcall-entity.yaml b/guide/blueprints/test/example_yaml/entities/testhttpcall-entity.yaml
new file mode 100644
index 0000000..e719b82
--- /dev/null
+++ b/guide/blueprints/test/example_yaml/entities/testhttpcall-entity.yaml
@@ -0,0 +1,7 @@
+- type: org.apache.brooklyn.test.framework.TestHttpCall
+  name: Check HTTP Response Status Code
+  url: $brooklyn:entity("tomcat").attributeWhenReady("webapp.url")
+  timeout: 60s
+  applyAssertionTo: status
+  assert:
+  - isEqualTo: 200
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/example_yaml/entities/testsensor-entity.yaml
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/example_yaml/entities/testsensor-entity.yaml b/guide/blueprints/test/example_yaml/entities/testsensor-entity.yaml
new file mode 100644
index 0000000..91f55a5
--- /dev/null
+++ b/guide/blueprints/test/example_yaml/entities/testsensor-entity.yaml
@@ -0,0 +1,7 @@
+- type: org.apache.brooklyn.test.framework.TestSensor
+  name: Check tomcat isUp
+  target: $brooklyn:entity("tomcat")
+  sensor: service.isUp
+  timeout: 10m
+  assert:
+  - equals: true
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/example_yaml/entities/testsshcommand-entity.yaml
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/example_yaml/entities/testsshcommand-entity.yaml b/guide/blueprints/test/example_yaml/entities/testsshcommand-entity.yaml
new file mode 100644
index 0000000..bcf39cf
--- /dev/null
+++ b/guide/blueprints/test/example_yaml/entities/testsshcommand-entity.yaml
@@ -0,0 +1,28 @@
+services:
+- type: org.apache.brooklyn.test.framework.TestCase
+  name: testcase1
+  brooklyn.children:
+    - type: org.apache.brooklyn.entity.webapp.tomcat.TomcatServer
+      id: testprocess
+
+    - type: org.apache.brooklyn.test.framework.TestSshCommand
+      name: Check tomcat process running with ps
+      targetId: testprocess
+      command: ps -ef
+      assertStatus:
+        equals: 0
+      assertOut:
+        contains: tomcat
+      assertErr: 
+        isEmpty: true
+
+    - type: org.apache.brooklyn.test.framework.TestSshCommand
+      name: Check hello world script
+      targetId: testprocess
+      downloadUrl: https://github.com/apache/brooklyn-docs/raw/master/guide/blueprints/test/example_yaml/entities/script1.sh
+      assertStatus:
+        equals: 0
+      assertOut: 
+        equals: hello world
+      assertErr: 
+        isEmpty: true

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/example_yaml/testcases/effector-test-snippet.yaml
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/example_yaml/testcases/effector-test-snippet.yaml b/guide/blueprints/test/example_yaml/testcases/effector-test-snippet.yaml
new file mode 100644
index 0000000..2e01ba5
--- /dev/null
+++ b/guide/blueprints/test/example_yaml/testcases/effector-test-snippet.yaml
@@ -0,0 +1,28 @@
+- type: org.apache.brooklyn.test.framework.TestCase
+  name: Check Deploy Effector
+  brooklyn.children:
+  - type: org.apache.brooklyn.test.framework.TestSensor
+    name: Check webappcluster isUp
+    targetId: webappcluster
+    sensor: service.isUp
+    timeout: 10m
+    assert:
+    - equals: true
+  - type: org.apache.brooklyn.test.framework.TestEffector
+    name: Invoke Deploy Effector
+    targetId: webappcluster
+    effector: deploy
+    timeout: 5m
+    params:
+      url: http://search.maven.org/remotecontent?filepath=org/apache/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.8.0-incubating/brooklyn-example-hello-world-sql-webapp-0.8.0-incubating.war
+      targetName: newcontext
+  - type: org.apache.brooklyn.test.framework.TestHttpCall
+    name: Check Deployed Webapp Status Code
+    timeout: 5m
+    url: >
+      $brooklyn:formatString("http://%s:%s/newcontext/",
+      $brooklyn:entity("webappcluster").attributeWhenReady("host.address"),
+      $brooklyn:entity("webappcluster").attributeWhenReady("proxy.http.port"))
+    applyAssertionTo: status
+    assert:
+    - isEqualTo: 200
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/example_yaml/testcases/getting-started-test-example.yaml
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/example_yaml/testcases/getting-started-test-example.yaml b/guide/blueprints/test/example_yaml/testcases/getting-started-test-example.yaml
new file mode 100644
index 0000000..d6740ec
--- /dev/null
+++ b/guide/blueprints/test/example_yaml/testcases/getting-started-test-example.yaml
@@ -0,0 +1,71 @@
+name: Getting Started Blueprint Test
+
+location:
+  jclouds:aws-ec2:
+    identity: ABCDEFGHIJKLMNOPQRST
+    credential: s3cr3tsq1rr3ls3cr3tsq1rr3ls3cr3tsq1rr3l
+
+services:
+- type: org.apache.brooklyn.entity.webapp.ControlledDynamicWebAppCluster
+  name: My Web
+  id: webappcluster
+  brooklyn.config:
+    wars.root: http://search.maven.org/remotecontent?filepath=org/apache/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.8.0-incubating/brooklyn-example-hello-world-sql-webapp-0.8.0-incubating.war
+    java.sysprops:
+      brooklyn.example.db.url: >
+        $brooklyn:formatString("jdbc:%s%s?user=%s&password=%s",
+        entity("db").attributeWhenReady("datastore.url"),
+        "visitors", "brooklyn", "br00k11n")
+- type: org.apache.brooklyn.entity.database.mysql.MySqlNode
+  id: db
+  name: My DB
+  brooklyn.config:
+    creationScriptUrl: https://bit.ly/brooklyn-visitors-creation-script
+- type: org.apache.brooklyn.test.framework.TestHttpCall
+  name: Check HTTP Response Status Code
+  url: >
+    $brooklyn:formatString("http://%s:%s",
+    $brooklyn:entity("webappcluster").attributeWhenReady("host.address"),
+    $brooklyn:entity("webappcluster").attributeWhenReady("proxy.http.port"))
+  timeout: 10m
+  applyAssertionTo: status
+  assert:
+  - isEqualTo: 200
+- type: org.apache.brooklyn.test.framework.TestHttpCall
+  name: Check HTTP Response Body
+  url: >
+    $brooklyn:formatString("http://%s:%s",
+    $brooklyn:entity("webappcluster").attributeWhenReady("host.address"),
+    $brooklyn:entity("webappcluster").attributeWhenReady("proxy.http.port"))
+  timeout: 10m
+  applyAssertionTo: body
+  assert:
+  - matches: "(?s).*Br[o]{2}klyn Deployed.*"
+- type: org.apache.brooklyn.test.framework.TestCase
+  name: Check Deploy Effector
+  brooklyn.children:
+  - type: org.apache.brooklyn.test.framework.TestSensor
+    name: Check webappcluster isUp
+    targetId: webappcluster
+    sensor: service.isUp
+    timeout: 10m
+    assert:
+    - equals: true
+  - type: org.apache.brooklyn.test.framework.TestEffector
+    name: Invoke Deploy Effector
+    targetId: webappcluster
+    effector: deploy
+    timeout: 5m
+    params:
+      url: http://search.maven.org/remotecontent?filepath=org/apache/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.8.0-incubating/brooklyn-example-hello-world-sql-webapp-0.8.0-incubating.war
+      targetName: newcontext
+  - type: org.apache.brooklyn.test.framework.TestHttpCall
+    name: Check Deployed Webapp Status Code
+    timeout: 5m
+    url: >
+      $brooklyn:formatString("http://%s:%s/newcontext/",
+      $brooklyn:entity("webappcluster").attributeWhenReady("host.address"),
+      $brooklyn:entity("webappcluster").attributeWhenReady("proxy.http.port"))
+    applyAssertionTo: status
+    assert:
+    - isEqualTo: 200
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/example_yaml/testcases/http-test-snippet.yaml
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/example_yaml/testcases/http-test-snippet.yaml b/guide/blueprints/test/example_yaml/testcases/http-test-snippet.yaml
new file mode 100644
index 0000000..0a7a953
--- /dev/null
+++ b/guide/blueprints/test/example_yaml/testcases/http-test-snippet.yaml
@@ -0,0 +1,20 @@
+- type: org.apache.brooklyn.test.framework.TestHttpCall
+  name: Check HTTP Response Status Code
+  url: >
+    $brooklyn:formatString("http://%s:%s/newcontext/",
+    $brooklyn:component("webappcluster").attributeWhenReady("host.address"),
+    $brooklyn:component("webappcluster").attributeWhenReady("proxy.http.port"))
+  timeout: 10m
+  applyAssertionTo: status
+  assert:
+  - equals: 200
+- type: org.apache.brooklyn.test.framework.TestHttpCall
+  name: Check HTTP Response Body
+  url: >
+    $brooklyn:formatString("http://%s:%s/newcontext/",
+    $brooklyn:component("webappcluster").attributeWhenReady("host.address"),
+    $brooklyn:component("webappcluster").attributeWhenReady("proxy.http.port"))
+  timeout: 10m
+  applyAssertionTo: body
+  assert:
+  - matches: "(?s).*Br[o]{2}klyn Deployed.*"
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/example_yaml/testcases/sensor-test-snippet.yaml
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/example_yaml/testcases/sensor-test-snippet.yaml b/guide/blueprints/test/example_yaml/testcases/sensor-test-snippet.yaml
new file mode 100644
index 0000000..da58cd1
--- /dev/null
+++ b/guide/blueprints/test/example_yaml/testcases/sensor-test-snippet.yaml
@@ -0,0 +1,7 @@
+- type: org.apache.brooklyn.test.framework.TestSensor
+  name: Check webappcluster isUp
+  targetId: webappcluster
+  sensor: service.isUp
+  timeout: 10m
+  assert:
+  - equals: true
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/images/getting-started-blueprint-test-large.png
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/images/getting-started-blueprint-test-large.png b/guide/blueprints/test/images/getting-started-blueprint-test-large.png
new file mode 100644
index 0000000..655e4f0
Binary files /dev/null and b/guide/blueprints/test/images/getting-started-blueprint-test-large.png differ

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/images/getting-started-blueprint-test.png
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/images/getting-started-blueprint-test.png b/guide/blueprints/test/images/getting-started-blueprint-test.png
new file mode 100644
index 0000000..09d23e8
Binary files /dev/null and b/guide/blueprints/test/images/getting-started-blueprint-test.png differ

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/index.md
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/index.md b/guide/blueprints/test/index.md
new file mode 100644
index 0000000..8b10f01
--- /dev/null
+++ b/guide/blueprints/test/index.md
@@ -0,0 +1,33 @@
+---
+title: Testing YAML Blueprints
+layout: website-normal
+children:
+- test-entities.md
+- usage-examples.md
+---
+
+Brooklyn provides a selection of test entities which can be used to validate Blueprints via YAML. The basic building block is a TargetableTestComponent, which is used to resolve a target. There are two different groups of entities that inherit from TargetableTestComponent. The first is structural, which effects how the tests are run, for example by affecting the order they are run in. The second group is validation, which is used to confirm the application is deployed as intended, for example by checking some sensor value.
+
+Structural test entities include:
+
+- `TestCase`  - starts child entities sequentially.
+- `ParallelTestCase` - starts child entities in parallel.
+- `LoopOverGroupMembersTestCase` - creates a TargetableTestComponent for each member of a group.
+- `InfrastructureDeploymentTestCase` - will create the specified Infrastructure and then deploy the target entity specifications there.
+
+Validation test entities include:
+
+- `TestSensor` - perform assertion on a specified sensor.
+- `TestEffector` - perform assertion on response to effector call.
+- `TestHttpCall` - perform assertion on response to specified HTTP GET Request.
+- `TestSshCommand` - test assertions on the result of an ssh command on the same machine as the target entity.
+- `TestWinrmCommand` - test assertions on the result of a WinRM command on the same machine as the target entity.
+- `TestEndpointReachable` - assert that a TCP endpoint is reachable. The endpoint can be in a 
+  number of different formats: a string in the form of `ip:port` or URI format; or a 
+  `com.google.common.net.HostAndPort` instance; or a `java.net.URI` instance; or a `java.net.URL` instance.
+
+TargetableTestComponents can be chained together, with the target being inherited by the components children. For example, a ParallelTestCase could be created that has a TestHttpCall as a child. As long as the TestHttpCall itself does not have a target, it will use the target of it's parent, ParallelTestCase. Using this technique, we can build up complex test scenarios.
+
+The following sections provide details on each test entity along with examples of their use.
+
+{% include list-children.html %}

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/test-entities.md
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/test-entities.md b/guide/blueprints/test/test-entities.md
new file mode 100644
index 0000000..37604a1
--- /dev/null
+++ b/guide/blueprints/test/test-entities.md
@@ -0,0 +1,198 @@
+---
+title: Blueprint Test Entities
+title_in_menu: Test Entities
+layout: website-normal
+---
+
+{% include fields.md %}
+
+
+## Structural Test Entities
+
+### TestCase
+The `TestCase` entity acts as a container for a list of child entities which are started *sequentially*.
+
+{% highlight yaml %}
+{% readj example_yaml/entities/testcase-entity.yaml %}
+{% endhighlight %}
+
+This can be used to enforce a strict ordering, for example ensuring a sensor has a certain value before attempting to invoke an effector.
+
+Timeouts on child entities should be set relative to the completion of the preceding entity.
+
+The `ParallelTestCase` entity can be added as a child to run a subset of entities in parallel as a single step.
+
+
+### ParallelTestCase
+The `ParallelTestCase` entity acts as a container for a list of child entities which are started in *parallel*.
+
+{% highlight yaml %}
+{% readj example_yaml/entities/paralleltestcase-entity.yaml %}
+{% endhighlight %}
+
+This can be used to run a subset of entities in parallel as a single step when nested under a `TestCase` entity.
+
+Timeouts on child entities should be set relative to the start of the `ParallelTestCase`.
+
+
+### LoopOverGroupMembersTestCase
+The `LoopOverGroupMembersTestCase` entity is configured with a target group and a test specification. For each member of the targeted group, the test case will create a TargetableTestComponent entity from the supplied test specification and set the components target to be the group member.
+
+{% highlight yaml %}
+{% readj example_yaml/entities/loopovergroupmembers-entity.yaml %}
+{% endhighlight %}
+
+#### Parameters
+- `target` - group who's members are to be tested, specified via DSL. For example, `$brooklyn:entity("tomcat")`. See also the `targetId` parameter.
+- `targetId` - alternative to the `target` parameter which wraps the DSL component lookup requiring only the `id` be supplied. For example, `tomcat`. Please note, this must point to a group.
+- `test.spec` - The TargetableTestComponent to create for each child.
+
+
+### InfrastructureDeploymentTestCase
+The `InfrastructureDeploymentTestCase` will first create and deploy an infrastructure from the `infrastructure.deployment.spec` config. It will then retrieve a deployment location by getting the value of the infrastructures `infrastructure.deployment.location.sensor` sensor. It will then create and deploy all entities from the `infrastructure.deployment.spec` config to the deployment location.
+
+{% highlight yaml %}
+{% readj example_yaml/entities/infrastructuredeploymenttestcase-entity.yaml %}
+{% endhighlight %}
+
+#### Parameters
+
+- `infrastructure.deployment.spec` - the infrastructure to be deployed.
+- `infrastructure.deployment.entity.specs` - the entities to be deployed to the infrastructure
+- `infrastructure.deployment.location.sensor` - the name of the sensor on the infrastructure to retrieve the deployment location
+
+
+## Validation Test Entities
+
+### TestSensor
+The `TestSensor` entity performs an assertion on a specified sensors value.
+
+{% highlight yaml %}
+{% readj example_yaml/entities/testsensor-entity.yaml %}
+{% endhighlight %}
+
+#### Parameters
+- `target` - entity whose sensor will be tested, specified via DSL. For example, `$brooklyn:entity("tomcat")`. See also the `targetId` parameter.
+- `targetId` - alternative to the `target` parameter which wraps the DSL component lookup requiring only the `id` be supplied. For example, `tomcat`.
+- `sensor` - sensor to evaluate. For example `service.isUp`.
+- `timeout` - duration to wait on assertion to return a result. For example `10s`, `10m`, etc
+- `assert` - assertion to perform on the specified sensor value. See section on assertions below.
+
+<div class="alert alert-info">
+    <strong>Tip:</strong> If the <code>TestSensor</code> is wrapped within a <code>TestCase</code>, 
+    <code>ParallelTestCase</code> or <code>LoopOverGroupMembersTestCase</code> that set the target, 
+    <strong>you don't need to specify the target</strong>, unless you want to test a different entity.
+</div>
+
+
+### TestEffector
+The `TestEffector` entity invokes the specified effector on a target entity. If the result of the effector is a String, it will then perform assertions on the result.
+{% highlight yaml %}
+{% readj example_yaml/entities/testeffector-entity.yaml %}
+{% endhighlight %}
+
+#### Parameters
+- `target` - entity whose effector will be invoked, specified via DSL. For example, `$brooklyn:entity("tomcat")`. See also the `targetId` parameter.
+- `targetId` - alternative to the `target` parameter which wraps the DSL component lookup requiring only the `id` be supplied. For example, `tomcat`.
+- `timeout` - duration to wait on the effector task to complete. For example `10s`, `10m`, etc
+- `effector` - effector to invoke, for example `deploy`.
+- `params` - parameters to pass to the effector, these will depend on the entity and effector being tested. The example above shows the `url` and `targetName` parameters being passed to Tomcats `deploy` effector.
+- `assert` - assertion to perform on the returned result. See section on assertions below.
+
+<div class="alert alert-info">
+    <strong>Tip:</strong> If the <code>TestEffector</code> is wrapped within a <code>TestCase</code>, 
+    <code>ParallelTestCase</code> or <code>LoopOverGroupMembersTestCase</code> that set the target, 
+    <strong>you don't need to specify the target</strong>, unless you want to test a different entity.
+</div>
+
+
+### TestHttpCall
+The `TestHttpCall` entity performs a HTTP GET on the specified URL and performs an assertion on the response.
+{% highlight yaml %}
+{% readj example_yaml/entities/testhttpcall-entity.yaml %}
+{% endhighlight %}
+
+#### Parameters
+- `url` - URL to perform GET request on, this can use DSL for example `$brooklyn:entity("tomcat").attributeWhenReady("webapp.url")`.
+- `timeout` - duration to wait on a HTTP response. For example `10s`, `10m`, etc
+- `applyAssertionTo` - The filed to apply the assertion to. For example `status`, `body`
+- `assert` - assertion to perform on the response.  See section on assertions below.
+
+<div class="alert alert-info">
+    <strong>Tip:</strong> If the <code>TestHttpCall</code> is wrapped within a <code>TestCase</code>, 
+    <code>ParallelTestCase</code> or <code>LoopOverGroupMembersTestCase</code> that set the target, 
+    <strong>you don't need to specify the target</strong>, unless you want to test a different entity.
+</div>
+
+
+### TestSshCommand
+The TestSshCommand runs a command on the host of the target entity.
+The script is expected not to run indefinitely, but to return a result (process exit code), along with its
+standard out and error streams, which can then be tested using assertions.
+If no assertions are explicitly configured, the default is to assert a non-zero exit code.
+
+Either a bash command may be provided in the YAML, or a URL for a script which will be executed.
+
+{% highlight yaml %}
+{% readj example_yaml/entities/testsshcommand-entity.yaml %}
+{% endhighlight %}
+
+#### Parameters
+- `command` - The shell command to execute. (This and `downloadUrl` are mutually exclusive.)
+- `downloadUrl` - URL for a script to download and execute. (This and `command` are mutually exclusive.)
+- `shell.env` - Map of environment variables to be set.
+- `scriptDir` - if `downloadUrl` is used.  The directory on the target host where downloaded scripts should be copied to.
+- `runDir` - the working directory where the command or script will be executed on the target host.
+- `assertStatus` - Assertions on the exit code of the command or script. See section on assertions below.
+- `assertOut` - Assertions on the standard output of the command as a String.
+- `assertErr` -  Assertions on the standard error of the command as a String.
+
+<div class="alert alert-info">
+    <strong>Tip:</strong> If the <code>TestSshCommand</code> is wrapped within a <code>TestCase</code>, 
+    <code>ParallelTestCase</code> or <code>LoopOverGroupMembersTestCase</code> that set the target, 
+    <strong>you don't need to specify the target</strong>, unless you want to test a different entity.
+</div>
+
+
+## Assertions
+
+The following conditions are provided by those test entities above that include assertions
+
+- `isNull` - asserts that the actual value is `null`.
+- `notNull` - asserts that the actual value is NOT `null`.
+- `isEqualTo` - asserts that the actual value equals an expected value.
+- `equalTo` - a synonym for `isEqualTo`
+- `equals` - a synonym for `isEqualTo`
+- `notEqual` - asserts that the actual value does not equal the expected value.
+- `matches` - asserts that the actual value matches a [regex pattern](http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html?is-external=true), for example `".*hello.*"`.
+  Note that regular expressions follow the Java defaults, e.g. for [line terminators](http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html#lt). 
+  One can use `"(?m)"` for multi-line mode (so that `^` and `$` also match just after/before line terminators, 
+  rather than just the start/end of the entire input sequence). The dotall mode, set with `"(?s)"`, can also 
+  be useful (so that `.` matches any character, including line terminators). 
+- `containsMatch` - asserts that the value contains a string that matches a regex. It follow the behaviour of 
+  the [Matcher.find() method](http://docs.oracle.com/javase/7/docs/api/java/util/regex/Matcher.html#find()).
+  This can be useful for simplifying matching in multi-line input.
+- `contains` - asserts that the actual value contains the supplied value
+- `isEmpty` - asserts that the actual value is an empty string
+- `notEmpty` - asserts that the actual value is a non empty string
+- `hasTruthValue` - asserts that the actual value has the expected interpretation as a boolean
+- `greaterThan` - asserts that the actual value is greater than the expected value according to Java's
+  [Comparable](https://docs.oracle.com/javase/8/docs/api/java/lang/Comparable.html) interface. Actual and
+  expected must be instances of the same type and implement `Comparable`.
+- `lessThan` - asserts that the actual value is less than the expected value according to Java's
+  [Comparable](https://docs.oracle.com/javase/8/docs/api/java/lang/Comparable.html) interface. Actual and
+  expected must be instances of the same type and implement `Comparable`.
+
+Assertions may be provided as a simple map:
+
+    assert:
+      contains: 2 users
+      matches: .*[\d]* days.*
+
+
+If there is the need to make multiple assertions with the same key, the assertions can be specified
+as a list of such maps:
+
+    assert:
+     - contains: 2 users
+     - contains: 2 days

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/test/usage-examples.md
----------------------------------------------------------------------
diff --git a/guide/blueprints/test/usage-examples.md b/guide/blueprints/test/usage-examples.md
new file mode 100644
index 0000000..08d88e3
--- /dev/null
+++ b/guide/blueprints/test/usage-examples.md
@@ -0,0 +1,58 @@
+---
+title: Example Blueprint Tests
+title_in_menu: Example Tests
+layout: website-normal
+---
+
+{% include fields.md %}
+
+## Introduction
+This section describes some simple tests based on the [Getting Started]({{ site.path.guide }}/start/blueprints.html#launching-from-a-blueprint) example blueprint:
+
+{% highlight yaml %}
+{% readj /guide/start/_my-web-cluster.yaml %}
+{% endhighlight %}
+
+The following sections contain yaml snippets that be appended to the list of services in the blueprint above, a complete blueprint is also provided [below](#full-example).
+
+Note that unless otherwise specified the following tests are executed in parallel with the deployment of the application services, all `timeout` values are set accordingly.
+
+### Sensor Test
+
+Demonstrates the following sensor assertion:
+
+- asserts that the `webappcluster` entity `service.isUp` sensor is `true` within 10 minutes of blueprint being deployed.
+
+{% highlight yaml %}
+{% read example_yaml/testcases/sensor-test-snippet.yaml %}
+{% endhighlight %}
+
+### HTTP Call Tests
+Demonstrates the following HTTP Call assertions against the specified `url`, which in these examples are being built from the `webappcluster` entities `host.address` and `proxy.http.port` sensors (the tester having flexibility in how the test URL is to be constructed):
+
+- asserts the response status code is 200 within 10 minutes of the blueprint being deployed.
+- asserts the response body matches the regex `(?s).*Br[o]{2}klyn Deployed.*` within 10 minutes of the blueprint being deployed. Note the presence of the `(?s)` dotall flag to test a multiline response.
+
+{% highlight yaml %}
+{% readj example_yaml/testcases/http-test-snippet.yaml %}
+{% endhighlight %}
+
+### Effector Test (via TestCase entity)
+
+This `TestEffector` example demonstrates the use of the `TestCase` and `TestSensor` entities to ensure the service has started before invoking an effector using the `TestEffector` entity.
+
+- `TestCase` entity starts its children sequentially
+  - asserts that the `webappcluster` entity `service.isUp` sensor is `true` within 10 minutes of the parent `TestCase` entity starting. Blocks start of the next child until it obtains a result (or times out).
+  - `deploy` effector invoked to deploy war to a `newcontext` with a 5 minute timeout to allow completion of the deploy task.
+  - asserts `/newcontext` url returns a HTTP status code 200 within 5 minutes of the effector being invoked (Note that this timeout is relative to the preceding test entity as they are being sequentially run as children of a `TestCase` entity).
+
+{% highlight yaml %}
+{% readj example_yaml/testcases/effector-test-snippet.yaml %}
+{% endhighlight %}
+
+### Full Example
+A sample blueprint containing all the tests described above is available [here](./example_yaml/testcases/getting-started-test-example.yaml).
+
+This blueprint will deploy the [Getting Started]({{ site.path.guide }}/start/blueprints.html#launching-from-a-blueprint) application and run all of the test entities, which if successful should appear in the web console as follows.
+
+[![Successful Getting Started App deployment and Test execution.](images/getting-started-blueprint-test.png)](images/getting-started-blueprint-test-large.png)

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/web-console-yaml-700.png
----------------------------------------------------------------------
diff --git a/guide/blueprints/web-console-yaml-700.png b/guide/blueprints/web-console-yaml-700.png
new file mode 100644
index 0000000..d6a2249
Binary files /dev/null and b/guide/blueprints/web-console-yaml-700.png differ

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/web-console-yaml.png
----------------------------------------------------------------------
diff --git a/guide/blueprints/web-console-yaml.png b/guide/blueprints/web-console-yaml.png
new file mode 100644
index 0000000..3d65a99
Binary files /dev/null and b/guide/blueprints/web-console-yaml.png differ

http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/1330dcd3/guide/blueprints/winrm/client.md
----------------------------------------------------------------------
diff --git a/guide/blueprints/winrm/client.md b/guide/blueprints/winrm/client.md
new file mode 100644
index 0000000..2a8e08a
--- /dev/null
+++ b/guide/blueprints/winrm/client.md
@@ -0,0 +1,125 @@
+---
+title: Winrm4j Client
+layout: website-normal
+---
+
+## Winrm4j parameters
+
+Check [org.apache.brooklyn.location.winrm.WinRmMachineLocation](https://github.com/apache/brooklyn-server/blob/master/software/winrm/src/main/java/org/apache/brooklyn/location/winrm/WinRmMachineLocation.java#L82-L112)
+parameters available for WinRM.
+
+* host <String>: Host to connect to (required).Default value `null`
+* port <Integer>: WinRM port to use when connecting to the remote machine.<br>
+  If no port is specified then it defaults to a port depending on the `winrm.useHttps` flag.
+* winrm.useHttps <Boolean>: The parameter tells the machine sensors whether the winrm port is over https. If the parameter is true then 5986 will be used as a winrm port.<br>
+  Default value: `false`
+* retriesOfNetworkFailures <Integer>: The parameter sets the number of retries for connection failures. If you use high value, consider taking care for the machine's network.<br>
+  Default value: `4`
+* winrm.useNtlm <Boolean>: The parameter configures tells the machine sensors whether the winrm port is over https. If the parameter is true then 5986 will be used as a winrm port.<br>
+  Default value: `true`
+* winrm.computerName <String>: Windows Computer Name to use for authentication.<br>
+  Default value: `null`
+* user <String>: User to connect as<br>
+  Default value: `null`
+* password <String>: Password to use to connect.<br>
+  Default value: `null`
+* waitWindowsToStart <Duration>: By default Brooklyn will return the machine immediately after Brooklyn is able to WinRM. Sometimes restart could happen after a Windows VM is provisioned.
+  This could be because of System Upgrade or other.
+  By setting this config key to 60s, 5m or other X Duration of time Brooklyn will wait X amount of time for disconnect to occur.
+  If connection failure occurs it will wait X amount of time for the machine to come up.<br>
+  Default value: `null`
+
+If there are location config keys prefixed with `brooklyn.winrm.config.` prefix will be removed
+and it will be used to instantiate a `org.apache.brooklyn.util.core.internal.winrm.WiRmTool` implementation.
+
+## WinRM Connectivity Diagnostics
+
+If you are experiencing problems with a windows blueprint against a jclouds location 
+where Apache Brooklyn complains about failing to connect to the IP you should check those things.
+
+1. Apache Brooklyn is using correct username and password
+1. Apache Brooklyn can reach the IP of the provisioned machine. WinRM port 5985 or 5986 is also reachable from Apache Brooklyn.
+1. Check whether `WinRmMachineLocation#getDefaultUserMetadataString(ConfigurationSupportInternal)` is applied on the VM.
+   This script should be passed to the cloud and executed in order to configure WinRM according to Apache Brooklyn requirements for authentication.
+   So far windows startup script are known to be supported on AWS EC2 and VCloud Director.
+   If your cloud doesn't use this script then tune WinRM parameters accordingly.
+1. Check whether you use winrm over http or over https.
+  1. If you are using WinRM over http then make sure WinRM service on target VM has `AllowUnencrypted = true`
+
+If the quick list above doesn't help then follow the steps bellow.
+
+To speed up diagnosing the problem we advice to trigger a deployment with the JcloudsLocation flag `destroyOnFailure: false` so you can check status of the provisioned machine
+or try later different WinRM parameters with a Apache Brooklyn [BYON Location](../../locations/index.html#byon).
+
+After you determined what is the username and the password you can proceed with next steps.
+*(Notice that for cloud providers which use Auto Generated password will not be logged.
+For these cases use Java Debug to retrieve ot or provision a VM manually with the same parameters when using Apache Brooklyn to provision a jclouds location.)*
+
+The first step is to find what is the winrm service configuration on the target host.
+
+1. If you have RDP access or KVM like access to the VM then check the winrm service status with the command bellow.
+   `winrm get winrm/config/service`
+   If you are using http you should have AllowUnencrypted to false.
+   Encryption is supported only over https.
+   Sample output:
+
+        MaxConcurrentOperations = 4294967295
+        MaxConcurrentOperationsPerUser = 1500
+        EnumerationTimeoutms = 240000
+        MaxConnections = 300
+        MaxPacketRetrievalTimeSeconds = 120
+        AllowUnencrypted = true
+        Auth
+            Basic = false
+            Kerberos = true
+            Negotiate = true
+            Certificate = false
+            CredSSP = true
+            CbtHardeningLevel = Relaxed
+        DefaultPorts
+            HTTP = 5985
+            HTTPS = 5986
+        IPv4Filter = *
+        IPv6Filter = *
+        EnableCompatibilityHttpListener = false
+        EnableCompatibilityHttpsListener = false
+        CertificateThumbprint
+        AllowRemoteAccess = true
+
+Use an Apache Brooklyn BYON blueprint to try easily other connection options.
+
+    location:
+      byon:
+        hosts:
+        - winrm: 10.0.0.1
+          user: Administrator
+          password: pa55w0rd
+          osFamily: windows
+    services:
+    - type: org.apache.brooklyn.entity.software.base.VanillaWindowsProcess
+      brooklyn.config:
+         checkRunning.command: echo checkRunning
+         install.command: echo installCommand
+
+1. Check IP is reachable from Apache Brooklyn instance
+   Check whether `telnet 10.0.0.1 5985` makes successfully a socket.
+1. If AllowUnencrypted is false and you are using winrm over http then apply `winrm set winrm/config/service @{AllowUnencrypted="true"}`
+   *If jclouds or the cloud provider doesn't support passing `sysprep-specialize-script-cmd` then consider modifying Windows VM Image.* 
+1. Check your username and password. Notice in Windows passwords are case sensitive.
+   Here is how it looks log from a wrong password:
+
+        INFO: Authorization loop detected on Conduit "{http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd}WinRmPort.http-conduit" on URL "http://192.168.122.1:5985/wsman" with realm "null"
+        Oct 21, 2016 10:43:11 AM org.apache.cxf.phase.PhaseInterceptorChain doDefaultLogging
+        WARNING: Interceptor for {http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd}WinRmService#{http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd}Create has thrown exception, unwinding now
+        org.apache.cxf.interceptor.Fault: Could not send Message.
+        at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:64)
+
+1. When having wrong password you may want to try logging on a different domain
+   This is possible from `brooklyn.winrm.config.winrm.computerName` location config.
+1. If you want to configure Windows target host with https then check the article [Configuring WINRM for HTTPS](https://support.microsoft.com/en-us/kb/2019527)
+1. If you are still seeing authorization errors then try connecting via winrm with the embedded winrs client.
+   First make sure you have the server in trusted hosts.
+
+Then execute a simple command like
+
+    winrs -r:10.0.2.15 -unencrypted -u:Administrator -p:pa55w0rd ipconfig