brooklyn-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rich...@apache.org
Subject [27/36] incubator-brooklyn git commit: Cherry-pick merge of PR #389 into 0.7.0-M2-incubating-docs branch
Date Fri, 09 Jan 2015 15:39:09 GMT
http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/201818b0/docs/guide/use/examples/global-web-fabric/index.md
----------------------------------------------------------------------
diff --git a/docs/guide/use/examples/global-web-fabric/index.md b/docs/guide/use/examples/global-web-fabric/index.md
new file mode 100644
index 0000000..b59eae0
--- /dev/null
+++ b/docs/guide/use/examples/global-web-fabric/index.md
@@ -0,0 +1,378 @@
+---
+layout: guide-normal
+title: Global Web Fabric
+toc: /guide/toc.json
+---
+
+This example shows how to build a multi-site web application *fabric*
+with DNS configured on the front-end to combine the sites,
+routing users to the location closest to them.
+
+It can combine with the [Simple Web Cluster](../webcluster) example
+or the [Portable Cloud Foundry](https://github.com/cloudsoft/brooklyn-cloudfoundry) example,
+but does not assume knowledge of either of these.
+
+{% readj ../before-begin.include.md %}
+
+Now, go to this particular example's directory:
+
+{% highlight bash %}
+% cd global-web-fabric
+{% endhighlight %}
+
+The CLI needs to know where to find your compiled examples. You can set this up by exporting
+the ``BROOKLYN_CLASSPATH`` environment variable in the following way:
+
+{% highlight bash %}
+% export BROOKLYN_CLASSPATH=$(pwd)/target/classes
+{% endhighlight %}
+
+The project ``global-web-fabric`` contains the code used
+in this example under ``src/main/java``.
+
+
+### Setting Up Geographic DNS
+
+This example uses [geoscaling.com](http://www.geoscaling.com) to provide **free** geographic-dependent DNS services.
+This will forward a domain name of your choice to various IPs depending on a script,
+e.g. computing the nearest IP based on latitude and longitude of the requester and the targets.
+Brooklyn will automatically generate and update this script, but you do need to 
+create and configure a Geoscaling account:
+
+ 1. Create the free account [here](https://www.geoscaling.com/dns2/?module=register).
+ 1. Click the link in the email you receive.
+ 1. Enter the domain name you wish to use into geoscaling (see below).
+
+The simplest domain name to choose is something unique under `geopaas.org`, e.g. `yourname.geopaas.org`,
+which we have already configured for Geoscaling to manage.
+If you are using your own domain name, 
+set its nameservers as advised by geoscaling (e.g. `ns{1,2,3,4}.geoscaling.com`).
+
+Next we need to supply this information to Brooklyn at runtime.
+The simplest way is to create or add the following fields to `~/.brooklyn/brooklyn.properties`:
+
+{% highlight bash %}
+brooklyn.geoscaling.username=yourname
+brooklyn.geoscaling.password=s3cr3t
+brooklyn.geoscaling.primaryDomain=yourname.geopaas.org
+{% endhighlight %}
+
+Replace the values of these fields as appropriate, of course!
+You can, if you prefer, supply (or override) these values in your Brooklyn application.
+
+
+### Setting Up the Locations Database
+
+In order to generate the "closest-IP" script,
+Brooklyn needs a way to find out the latitude and longitude of the
+servers you are using.
+The simplest way to do this is do download the free GeoCityLite binary flatfile 
+from [MaxMind](http://dev.maxmind.com/geoip/geoip2/geolite2/#Downloads),
+unpack it, and copy it to `~/.brooklyn/GeoLite2-City.mmdb`.
+
+This will be picked up automatically if it is installed.
+You can instead specify to use an online lookup service, such as 
+[utrace.de](http://www.utrace.de) by specifying
+`-Dbrooklyn.location.geo.HostGeoLookup=brooklyn.location.geo.UtraceHostGeoLookup`;
+but note this has a cap of 100 per day.
+
+This information is also used to display locations on the map
+in the Brooklyn dashboard.
+Note however that these free services are not 100% accurate;
+they are handy for dev/test but in a production system
+you may wish to specify the geographical information manually in your application,
+or purchase a commercial locations-database subscription.
+
+
+## The Code
+
+Now let's start writing our application.
+The heavy lifting will be done by off-the-shelf Brooklyn classes:
+
+ * `DynamicFabric` will create the entity specified by `factory` in each location it is given
+ * `GeoscalingDnsService` monitors children of a specified entity (the `DynamicFabric`) 
+   and adds them as DNS targets for the region they are in  
+
+First, however, let's create the Java class -- call it `GlobalWebFabricExample`.
+This will extend the Brooklyn `AbstractApplication`:
+
+{% highlight java %}
+package brooklyn.demo;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+import brooklyn.entity.basic.AbstractApplication;
+
+public class GlobalWebFabricExample extends AbstractApplication {
+    @Override
+    public void init() {
+         // TODO create our app!
+    }
+}
+{% endhighlight %}
+
+### The Fabric
+
+The `DynamicFabric` by default has no knowledge of what it will build,
+other than the `factory` it is given to create an entity in each region.
+We'll use the class `ElasticJavaWebAppService.Factory` which creates
+an elastic Java Web App service,
+such as the `ControlledDynamicWebAppCluster` used in the
+[Simple Web Cluster](../webcluster) example, if deploying to VMs,
+or perhaps a `CloudFoundryJavaWebAppCluster` if deploying to a Cloud Foundry location
+(see [brooklyn-cloudfoundry repo](https://github.com/cloudsoft/brooklyn-cloudfoundry)).
+
+{% highlight java %}
+        DynamicFabric webFabric = addChild(EntitySpec.create(DynamicFabric.class)
+                .displayName("Web Fabric")
+                .configure(DynamicFabric.FACTORY, new ElasticJavaWebAppService.Factory())
+                .configure(ElasticJavaWebAppService.ROOT_WAR, WAR_PATH));
+{% endhighlight %}
+
+Here we have specified the WAR to use with `configure(ElasticJavaWebAppService.ROOT_WAR, WAR_PATH)`.
+The war configuration used in the previous example is only available on web-aware entities;
+configuration specified with a ConfigKey can be done on any entity,
+and is inherited at runtime, so this provides a useful way to specify the WAR to use
+even though the web-aware entities are only constructed at runtime.
+
+
+### Stitching the Fabric together with DNS
+
+To stitch these together seamlessly, another entity will run a policy
+which collects the public-facing IP address of each cluster created by the fabric,
+as it comes online, by watching for `SERVICE_UP` sensors.
+First, however, let's make sure any load-balancer proxies (e.g. nginx) in these clusters
+are listening on port 80:
+
+{% highlight java %}
+        DynamicFabric webFabric = addChild(EntitySpec.create(DynamicFabric.class)
+                .displayName("Web Fabric")
+                .configure(DynamicFabric.FACTORY, new ElasticJavaWebAppService.Factory())
+                .configure(ElasticJavaWebAppService.ROOT_WAR, WAR_PATH)
+                .configure(AbstractController.PROXY_HTTP_PORT, PortRanges.fromInteger(80)));
+{% endhighlight %}
+
+Let's now define the Geoscaling entity which does the stitching.
+We need to supply the username, password, and primaryDomainName for Geoscaling;
+we'll take this from the `brooklyn.properties` file mentioned above.
+We'll also specify a `smartSubdomainName`, to use Geoscaling's facility for
+lightweight sub-domains to prevent DNS caching and multiple instances of our application
+from confusing us -- e.g. `brooklyn-1234.yourname.geopaas.org`.
+
+{% highlight java %}
+        StringConfigMap config = getManagementContext().getConfig();
+        
+        GeoscalingDnsService geoDns = addChild(EntitySpec.create(GeoscalingDnsService.class)
+                .displayName("GeoScaling DNS")
+                .configure("username", checkNotNull(config.getFirst("brooklyn.geoscaling.username"), "username"))
+                .configure("password", checkNotNull(config.getFirst("brooklyn.geoscaling.password"), "password"))
+                .configure("primaryDomainName", checkNotNull(config.getFirst("brooklyn.geoscaling.primaryDomain"), "primaryDomain")) 
+                .configure("smartSubdomainName", "brooklyn"));
+{% endhighlight %}
+
+Lastly we need to tell this instance what entity it should monitor
+for children to include as targets:
+
+{% highlight java %}
+        geoDns.setTargetEntityProvider(webFabric);
+{% endhighlight %}
+
+
+
+### Cloud Foundry and other PaaS Targets
+
+At this point our core application is ready, and can be deployed to AWS or another VM cloud.
+This may take between 15 and 30 minutes to run,
+mainly spent downloading software
+(unless of course you specify a pre-configured `imageId` which contains the software).
+
+A quicker alternative is to deploy to a Java Web App platform-as-a-service
+such as Cloud Foundry.  A major advantage here is that they can provision very quickly,
+in a matter of seconds.  Code for this can be found in the 
+[brooklyn-cloudfoundry repo](https://github.com/cloudsoft/brooklyn-cloudfoundry), 
+along with an example global-web-fabric app.
+
+
+### Imports
+
+Your imports should look as follows:
+
+{% highlight java %}
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import brooklyn.config.StringConfigMap;
+import brooklyn.entity.basic.AbstractApplication;
+import brooklyn.entity.basic.Attributes;
+import brooklyn.entity.dns.geoscaling.GeoscalingDnsService;
+import brooklyn.entity.group.DynamicFabric;
+import brooklyn.entity.proxy.AbstractController;
+import brooklyn.entity.proxying.EntitySpec;
+import brooklyn.entity.webapp.ElasticJavaWebAppService;
+import brooklyn.location.basic.PortRanges;
+{% endhighlight %}
+
+
+### Use of main method (optional)
+
+In this example, we will use the brooklyn CLI launcher. However, it is possible to write your own main method.
+
+The following static constants are assumed (most of these as in the [Simple Web Cluster](../webcluster) example and others): 
+
+ * `WAR_PATH`, pointing to the webapp to deploy (a default supplied as part of the Brooklyn examples is used here)
+ * `DEFAULT_LOCATIONS`, containing a string spec of the locations to deploy to if none are supplied on the command-line;
+   for this example `localhost` will frequently not work unless Geoscaling can see it 
+   (i.e. it has a public IP and appropriate firewall settings)
+
+The code (which can safely be omitted) is as follows:
+
+{% highlight java %}
+import brooklyn.launcher.BrooklynLauncher;
+import brooklyn.util.CommandLineUtil;
+
+import com.google.common.base.Joiner;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+
+// class definition, and rest of class goes here...
+
+    public static final Logger log = LoggerFactory.getLogger(GlobalWebFabricExample.class);
+    
+    // points to the webapp to deploy (a default supplied as part of the Brooklyn examples is used here)
+    public static final String WAR_PATH = "classpath://hello-world-webapp.war";
+    
+    // locations to deploy to if none are supplied on the command-line; for this example `localhost` will 
+    // frequently not work unless Geoscaling can see it (i.e. it has a public IP and appropriate firewall settings)
+    static final List<String> DEFAULT_LOCATIONS = ImmutableList.of(
+            "aws-ec2:eu-west-1",
+            "aws-ec2:ap-southeast-1",
+            "aws-ec2:us-west-1" );
+        
+    public static void main(String[] argv) {
+        List<String> args = Lists.newArrayList(argv);
+        String port =  CommandLineUtil.getCommandLineOption(args, "--port", "8081+");
+        String locations = CommandLineUtil.getCommandLineOption(args, "--locations", Joiner.on(",").join(DEFAULT_LOCATIONS));
+
+        BrooklynLauncher launcher = BrooklynLauncher.newInstance()
+                .application(EntitySpec.create(StartableApplication.class, GlobalWebFabricExample.class).displayName("Brooklyn Global Web Fabric Example"))
+                .webconsolePort(port)
+                .locations(Arrays.asList(locations))
+                .start();
+        
+        Entities.dumpInfo(app);
+    }
+{% endhighlight %}
+
+
+
+## Running the Example
+
+Now let's run this example.
+
+{% highlight bash %}
+${BROOKLYN_HOME}/bin/brooklyn launch --app brooklyn.demo.GlobalWebFabricExample \
+--location jclouds:aws-ec2:eu-west-1,jclouds:aws-ec2:ap-southeast-1,jclouds:aws-ec2:us-west-1 
+{% endhighlight %}
+
+The management web console will start,
+followed by the web-app services in the locations specified
+creating the VM's as needed.
+Let's look at the management web console, on port 8081:
+
+[![Web Console Map](console-map-w700.png "Web Console Map")](console-map.png) 
+
+This shows the targets (e.g. Ireland (AWS eu-west-1),  Singapore (AWS ap-southeast-1),  and California (AWS us-west-1)).
+This also shows the current status of the application.
+
+Navigating to the "applications" tab, we can view sensors, invoke effectors, control policies,
+and track activity, 
+for instance if a cluster is slow to start and you want to find out what is going on
+(you'll find additional information in the `brooklyn.log` file).
+Let's drill down on the Geoscaling DNS entity's sensors:
+
+[![Web Console Geoscaling Details](console-geoscaling-details-w700.png "Web Console Geoscaling Details")](console-geoscaling-details.png)
+
+Here we see it has chosen `brooklyn-vOZ7b4BL.martincloudsoft.geopaas.org` as the geo-load-balanced domain name.
+(Yours will be under `yourname.geopaas.org`, unless you chose a different domain earlier.)
+We can also see the hosts it is forwarding to, one for each cluster, corresponding to the
+children of the Web Fabric (propagated from the nginx hostnames, in the case of the ControlledDynamicWebAppCluster instances).
+
+
+### Checking the Web App
+
+Once Geoscaling reports at least one target, you should be able to access it on the geo-load-balanced domain name:
+
+[![Our Deployed Application](geopaas-deployed-app-w700.png "Our Deployed Application")](geopaas-deployed-app.png)
+
+Under the covers you are being routed to one of the clusters that has been deployed --
+whichever one is closest to you.
+(Due to DNS caching, at your machine or your ISP, clusters which come online after your first lookup
+will not be picked up until TTL expires, typically 10m, although often more if DNS services don't respect TTL.)
+
+
+### Checking DNS Information
+
+Let's find out exactly where we were routed:
+
+{% highlight bash %}
+% dig brooklyn-csgFCzTm.geopaas.org
+
+; <<>> DiG 9.4.3-P3 <<>> brooklyn-csgFCzTm.geopaas.org
+
+;; QUESTION SECTION:
+;brooklyn-csgFCzTm.geopaas.org. IN      A
+
+;; ANSWER SECTION:
+brooklyn-csgFCzTm.geopaas.org. 120 IN   CNAME   ec2-46-137-138-4.eu-west-1.compute.amazonaws.com.
+ec2-46-137-138-4.eu-west-1.compute.amazonaws.com. 215 IN A 46.137.138.4
+{% endhighlight %}
+
+This was run from Scotland so it seems a sensible choice.
+(Some portions of the output from `dig` have been removed for readability.)
+
+We can get more information by looking at the TXT records: 
+
+{% highlight bash %}
+% dig +trace @ns1.geoscaling.com TXT brooklyn-csgFCzTm.geopaas.org
+
+; <<>> DiG 9.4.3-P3 <<>> +trace @ns1.geoscaling.com TXT brooklyn-csgFCzTm.geopaas.org
+
+...
+
+geopaas.org.            86400   IN      NS      ns1.geoscaling.com.
+geopaas.org.            86400   IN      NS      ns2.geoscaling.com.
+geopaas.org.            86400   IN      NS      ns3.geoscaling.com.
+geopaas.org.            86400   IN      NS      ns4.geoscaling.com.
+;; Received 133 bytes from 199.249.112.1#53(a2.org.afilias-nst.info) in 45 ms
+
+brooklyn-csgFCzTm.geopaas.org. 300 IN   TXT     "Request from [54,-2]-(GB) directed to Ireland (IE)"
+brooklyn-csgFCzTm.geopaas.org. 300 IN   TXT     "GeoScaling config auto-updated by Brooklyn 2012-04-26 12:27:25 UTC"
+;; Received 189 bytes from 80.87.128.195#53(ns3.geoscaling.com) in 60 ms
+{% endhighlight %}
+
+
+## Next Steps
+
+This example has shown how to create a multi-region fabric, using the abstractions from
+[jclouds](http://jclouds.org) under the covers to make it easy to access different hosting providers
+simultaneously, and using higher-level abstractions in Brooklyn to mix PaaS systems with
+bare-VM (or even bare-metal, if you specify fixed IPs).
+
+This is meant as just the beginning however.  
+Here are some questions to think about and code challenges to give you a steer for what to explore next.
+
+
+ 1. The routines used at Geoscaling optimize for latency between the user and the location of the web-cluster.
+    What other strategies might be used?  Cost?  Compliance?  How would you code these?
+    
+ 2. This example ignores data, but you clearly can't do that in the real world.
+    When big-data is involved, does this introduce other considerations for optimizing geo-location?
+    
+ 3. Add a data tier to this system, such as MySQL or Mongo, or even Hadoop.
+    You might start with a single instance or cluster,
+    but the real fun starts with a fabric, and defining the synchronization/replication strategies
+    between the different clusters.
+    This isn't for the faint-hearted, but whatever you create will certainly be of interest
+    to people in the Brooklyn community.
+    Please [let us know]({{ site.path.guide }}/meta/contact.html) what you've built!

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/201818b0/docs/guide/use/examples/index.md
----------------------------------------------------------------------
diff --git a/docs/guide/use/examples/index.md b/docs/guide/use/examples/index.md
new file mode 100644
index 0000000..f55643f
--- /dev/null
+++ b/docs/guide/use/examples/index.md
@@ -0,0 +1,18 @@
+---
+layout: guide-normal
+title: Examples
+toc: /guide/toc.json
+---
+
+We currently have the following examples on the site:
+
+{% capture ltocs %}{% readj toc.json %}{% endcapture %}
+{% jsonball ltoc from var ltocs %}
+
+{% for x in ltoc %}
+* <a href="{{ x.file }}">{{ x.title }}</a>
+{% endfor %} 
+
+There are examples in the code also, just check out the examples/ project.
+
+**Have one of your own?**  [Add it here!]({{site.path.guide}}/dev/tips/update-docs.html)

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/201818b0/docs/guide/use/examples/messaging/index.md
----------------------------------------------------------------------
diff --git a/docs/guide/use/examples/messaging/index.md b/docs/guide/use/examples/messaging/index.md
new file mode 100644
index 0000000..c3c2254
--- /dev/null
+++ b/docs/guide/use/examples/messaging/index.md
@@ -0,0 +1,181 @@
+---
+layout: guide-normal
+title: Publish-Subscribe Messagiung
+toc: /guide/toc.json
+---
+
+This example shows how a simple messaging application can be build
+in brooklyn, starting with configuring and launching a broker. For
+these examples we will use the Apache [Qpid](http://qpid.apache.org/)
+Java AMQP message broker and clients using the
+[JMS](http://docs.oracle.com/javaee/6/tutorial/doc/bnceh.html) API.
+
+{% readj ../before-begin.include.md %}
+
+Now, go to this particular example's directory:
+
+{% highlight bash %}
+% cd simple-messaging-pubsub
+{% endhighlight %}
+
+The CLI needs to know where to find your compiled examples. You can set this up by exporting
+the ``BROOKLYN_CLASSPATH`` environment variable in the following way:
+
+{% highlight bash %}
+% export BROOKLYN_CLASSPATH=$(pwd)/target/classes
+{% endhighlight %}
+
+The project ``simple-messaging-pubsub`` includes a deployment
+descriptor for our example messaging application and simple _Publish_
+and _Subscribe_ JMS test client scripts.
+
+## Single Broker
+
+The first example will include a Qpid broker, which we will customize
+to use the Oracle [BDB](http://www.oracle.com/technetwork/products/berkeleydb/overview/index.html)
+message store as an example of a typical production setup. We will
+also create a queue for use by a pair of test clients.
+
+The ``QpidBroker`` entity is created like this, which uses the
+default configuration, specifying only the AMQP port and creates
+no queues or topics:
+
+{% highlight java %}
+public class StandaloneQpidBrokerExample extends AbstractApplication {
+    @Override
+    public void init() {
+        // Configure the Qpid broker entity
+    	QpidBroker broker = addChild(EntitySpec.create(QpidBroker.class)
+    	        .configure("amqpPort", 5672));
+    }
+}
+{% endhighlight %}
+
+To install the custom configuration files and extra libraries for
+BDB, we specify some files to copy to the broker installation, using
+the ``runtimeFiles`` property. These files should be available in
+the classpath of the application when it is running, usually by
+copying them to the ``src/main/resources`` directory. For example,
+here we copy a custom XML configuration file and a new password
+file:
+
+{% highlight java %}
+        final String CUSTOM_CONFIG_PATH = "classpath://custom-config.xml";
+        final String PASSWD_PATH = "classpath://passwd";
+
+    	QpidBroker broker = addChild(EntitySpec.create(QpidBroker.class)
+    	        .configure("amqpPort", 5672)
+    	        .configure("amqpVersion", AmqpServer.AMQP_0_10)
+    	        .configure("runtimeFiles", ImmutableMap.builder()
+    	                .put(QpidBroker.CONFIG_XML, CUSTOM_CONFIG_PATH)
+    	                .put(QpidBroker.PASSWD, PASSWD_PATH)
+    	                .build()));
+{% endhighlight %}
+
+Finally, we come to the complete configuration of our ``QpidBroker``
+entity using the BDB store. The additional properties here specify
+the AMQP version and that a queue named _testQueue_ should be created
+on startup.
+
+{% highlight java %}
+        final String CUSTOM_CONFIG_PATH = "classpath://custom-config.xml";
+        final String PASSWD_PATH = "classpath://passwd";
+        final String QPID_BDBSTORE_JAR_PATH = "classpath://qpid-bdbstore-0.14.jar";
+        final String BDBSTORE_JAR_PATH = "classpath://je-5.0.34.jar";
+
+    	QpidBroker broker = addChild(EntitySpec.create(QpidBroker.class)
+    	        .configure("amqpPort", 5672)
+    	        .configure("amqpVersion", AmqpServer.AMQP_0_10)
+    	        .configure("runtimeFiles", ImmutableMap.builder()
+    	                .put(QpidBroker.CONFIG_XML, CUSTOM_CONFIG_PATH)
+    	                .put(QpidBroker.PASSWD, PASSWD_PATH)
+    	                .put("lib/opt/qpid-bdbstore-0.14.jar", QPID_BDBSTORE_JAR_PATH)
+    	                .put("lib/opt/je-5.0.34.jar", BDBSTORE_JAR_PATH)
+    	                .build())
+    	        .configure("queue", "testQueue"));
+{% endhighlight %}
+
+
+### Running the Example
+
+You can run the example as follows:
+
+{% highlight bash %}
+% ${BROOKLYN_HOME}/bin/brooklyn -v launch --app brooklyn.demo.StandaloneQpidBrokerExample --location localhost
+{% endhighlight %}
+
+Now, visit the Brooklyn web console on port 8081 (for pre 0.6 releases,
+use the credentials admin/password). This allows you to view the Brooklyn 
+entities and their current state for debugging.
+
+Note that the installation may take some time, because the default
+deployment downloads the software from the official repos.  You can
+monitor start-up activity for each entity in the ``Activity`` pane
+in the management console, and see more detail by tailing the log
+file (``tail -f brooklyn.log``).
+
+After starting up, the demo script should display a summary of all
+the Brooklyn managed entities and their attributes. This will show
+both the Qpid broker and its child entity, the queue _testQueue_
+which was created at startup. The queue entity has sensors that
+monitor the depth of unread messages, which you can check while
+running the test client scripts later.
+
+If the ``-v`` flag is passed to the startup command, all configured
+entity and sensor details will be output. This includes the broker URL,
+which is used to configure JMS clients to connect to this broker.
+This URL can also be viewed as a sensor attribute in the web console,
+named _broker.url_.
+
+This sensor is common to _all_ messaging brokers that Brooklyn
+provides, and is usually accessed by applications to allow them to
+provide it as a parameter to other entities, as shown in the code
+fragment below.
+
+{% highlight java %}
+String url = broker.getAttribute(MessageBroker.BROKER_URL)
+{% endhighlight %}
+
+Using the URL the demo script printed, you can run the test ``Subscribe``
+and then ``Publish`` classes, to send messages using the broker. Simply
+run the commands in another window, with the provided URL as the
+only argument. Note that the URLs may be different to those printed
+below, and that any unquoted ``&`` characters *must* be escaped,
+if present.
+
+{% highlight bash %}
+% URL="amqp://guest:guest@/localhost?brokerlist='tcp://localhost:5672'"
+% java -cp "./resources/lib/*:./target/classes" brooklyn.demo.Subscribe ${URL}
+% java -cp "./resources/lib/*:./target/classes" brooklyn.demo.Publish ${URL}
+{% endhighlight %}
+
+In the _Publish_ window you should see a log message every time a
+message is sent, like this:
+
+{% highlight bash %}
+2012-05-02 14:04:38,521 INFO  Sent message 65
+2012-05-02 14:04:39,522 INFO  Sent message 66
+{% endhighlight %}
+
+Similarly, the _Subscribe_ windows should log on reciept of these
+messages, as follows:
+
+{% highlight bash %}
+2012-05-02 14:04:32,522 INFO  got message 41 test message 41
+2012-05-02 14:04:33,523 INFO  got message 42 test message 42
+{% endhighlight %}
+
+### Cloud Deployment
+
+With appropriate setup (as described
+[here]({{ site.path.guide }}/use/guide/management/index.html#startup-config))
+this can also be deployed to your favourite cloud, let's pretend
+it's Amazon Ireland, as follows:
+
+{% highlight bash %}
+% ${BROOKLYN_HOME}/bin/brooklyn launch --app brooklyn.demo.StandaloneQpidBrokerExample --location aws-ec2:eu-west-1
+{% endhighlight %}
+
+If you encounter any difficulties, please
+[tell us]({{ site.path.guide }}/meta/contact.html) and we'll do our best
+to help.

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/201818b0/docs/guide/use/examples/nosql-cassandra/cassandra.include.md
----------------------------------------------------------------------
diff --git a/docs/guide/use/examples/nosql-cassandra/cassandra.include.md b/docs/guide/use/examples/nosql-cassandra/cassandra.include.md
new file mode 100644
index 0000000..56319f8
--- /dev/null
+++ b/docs/guide/use/examples/nosql-cassandra/cassandra.include.md
@@ -0,0 +1,282 @@
+      
+{% readj ../before-begin.include.md %}
+
+## Simple Cassandra Cluster
+
+Go to this particular example's directory:
+
+{% highlight bash %}
+% cd simple-nosql-cluster
+{% endhighlight %}
+
+The CLI needs to know where to find your compiled examples. You can set this up by exporting
+the ``BROOKLYN_CLASSPATH`` environment variable in the following way:
+
+{% highlight bash %}
+% export BROOKLYN_CLASSPATH=$(pwd)/target/classes
+{% endhighlight %}
+
+The project ``simple-nosql-cluster`` includes several deployment descriptors
+for deploying and managing Cassandra, under ``src/main/java``.
+
+The simplest of these, ``SimpleCassandraCluster``, will start a Cassandra cluster. The code is:
+
+{% highlight java %}
+public class SimpleCassandraCluster extends AbstractApplication {
+  public void init() {
+    addChild(EntitySpec.create(CassandraCluster.class)
+        .configure(CassandraCluster.INITIAL_SIZE, 1)
+        .configure(CassandraCluster.CLUSTER_NAME, "Brooklyn"));
+  }
+}
+{% endhighlight %}
+
+To run that example on localhost (on *nix or Mac, assuming `ssh localhost` requires no password or passphrase):
+
+{% highlight bash %}
+% ${BROOKLYN_HOME}/bin/brooklyn launch --app brooklyn.demo.SimpleCassandraCluster \
+  --location localhost
+{% endhighlight %}
+
+Then visit the Brooklyn console on ``localhost:8081``.
+Note that the installation may take some time, because the default deployment downloads the software from
+the official repos.  You can monitor start-up activity for each entity in the ``Activity`` pane in the management console,
+and see more detail by tailing the log file (``tail -f brooklyn.log``).
+
+This example runs successfully on a local machine because ``INITIAL_SIZE`` is configured to just one node
+(a limitation of Cassandra is that every node must be on a different machine/VM).
+If you want to run with more than one node in the cluster, you'll need to use a location 
+that either points to multiple existing machines or to a cloud provider where you can 
+provision new machines.
+
+With appropriate setup of credentials (as described [here]({{ site.path.guide }}/use/guide/management/index.html#startup-config)) 
+this example can also be deployed to your favourite cloud. Let's pretend it's Amazon US East, as follows: 
+
+{% highlight bash %}
+% ${BROOKLYN_HOME}/bin/brooklyn launch --app brooklyn.demo.SimpleCassandraCluster \
+  --location aws-ec2:us-east-1
+{% endhighlight %}
+
+If you want more nodes in your cluster, you can either modify the deployment descriptor (i.e. change the ``INITIAL_SIZE`` value),
+or dynamically add more nodes by calling the ``resize`` effector through the web-console. 
+To do the latter, select cluster entity in the tree on the left, then click on the "effectors" tab, and invoke ``resize`` 
+with the desired number of nodes.
+
+
+### Testing your Cluster
+
+An easy way to test your cluster is to use the ``cassandra-stress`` command line tool.
+For example, run:
+
+{% highlight bash %}
+# Substitute the id below for your VM
+NODE_IDS=ec2-54-221-69-95.compute-1.amazonaws.com
+/tmp/brooklyn-aled/installs/CassandraNode/1.2.9/apache-cassandra-1.2.9/tools/bin/cassandra-stress \
+	--nodes ${NODE_IDS} \
+    --replication-factor 1 \
+    --progress-interval 1 \
+    --num-keys 10000 \
+    --operation INSERT
+{% endhighlight %}
+
+This command will fire 10000 inserts at the cluster, via the nodes specified in the comma-separated node list. 
+If you change ``INSERT`` to ``READ``, it will read each of those 10000 values.
+
+
+## High Availability Cassandra Cluster
+
+Ready for something more interesting?  Try this:
+
+{% highlight bash %}
+% ${BROOKLYN_HOME}/bin/brooklyn launch --app brooklyn.demo.HighAvailabilityCassandraCluster \
+  --location aws-ec2:us-east-1
+{% endhighlight %}
+
+This launches the class ``HighAvailabilityCassandraCluster``,
+which launches a Cassandra cluster configured to replicate across availability zones.
+
+To give some background for that statement, in 
+[AWS](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html)
+(and various other clouds), a region is a 
+separate geographic area, consisting of multiple isolated locations known as availability zones.
+To ensure high availability, the Cassandra cluster and thus the data should be spread across the 
+availability zones. Cassandra should be configured to ensure there is at least one replica in
+each availability zone. In 
+[Cassandra terminology](http://www.datastax.com/docs/1.1/cluster_architecture/replication)
+a region is normally mapped to a "datacenter" and an availability zone to a "rack".
+
+To be properly highly available, we need some automated policies to restart failed servers 
+and to replace unhealthy nodes. Brooklyn has these policies available out-of-the-box.
+To wire them up, the essential code fragment looks like this:
+
+{% highlight java %}
+public class HighAvailabilityCassandraCluster extends AbstractApplication {
+  public void init() {
+    addChild(EntitySpec.create(CassandraCluster.class)
+        .configure(CassandraCluster.CLUSTER_NAME, "Brooklyn")
+        .configure(CassandraCluster.INITIAL_SIZE, 1)
+        .configure(CassandraCluster.ENABLE_AVAILABILITY_ZONES, true)
+        .configure(CassandraCluster.NUM_AVAILABILITY_ZONES, 3)
+        .configure(CassandraCluster.ENDPOINT_SNITCH_NAME, "GossipingPropertyFileSnitch")
+        .configure(CassandraCluster.MEMBER_SPEC, EntitySpec.create(CassandraNode.class)
+            .policy(PolicySpec.create(ServiceFailureDetector.class))
+            .policy(PolicySpec.create(ServiceRestarter.class)
+                .configure(ServiceRestarter.FAILURE_SENSOR_TO_MONITOR, ServiceFailureDetector.ENTITY_FAILED)))
+        .policy(PolicySpec.create(ServiceReplacer.class)
+            .configure(ServiceReplacer.FAILURE_SENSOR_TO_MONITOR, ServiceRestarter.ENTITY_RESTART_FAILED)));
+  }
+}
+{% endhighlight %}
+
+This code is doing a lot and deserves some more detailed explanation:
+
+* The ``MEMBER_SPEC`` describes the configuration of the Cassandra nodes to be created in the cluster.
+  Assuming you're happy to use all the default thrift port etc, then the only configuration to add is
+  a couple of policies.
+* The ``ServiceFailureDetector`` policy watches the node's sensors, and generates
+  an ``ENTITY_FAILED`` event if the node goes down.
+* The ``ServiceRestarter`` policy responds to this failure-event
+  by restarting the node. Its default configuration is that: if a node does not come back up, or if it 
+  fails again within three minutes, then it will emit an ``ENTITY_RESTART_FAILED`` event.
+* Finally, the ``SERVICE_REPLACER`` policy on the cluster responds to this event by replacing the
+  entire VM. It sets up a new VM in the same location, and then tears down the faulty node.
+
+> *Troubleshooting:*
+
+> *In AWS, some availability zones can be constrained for particular instance sizes (see
+  [this bug report](https://github.com/brooklyncentral/brooklyn/issues/973)
+  If you get this error, the workaround is to specify explicitly the availability zones to use. 
+  This requires an additional line of code such as:*
+
+{% highlight java %}
+  .configure(AVAILABILITY_ZONE_NAMES, ImmutableList.of("us-east-1b", "us-east-1c", "us-east-1e"))
+{% endhighlight %}
+
+> *However, this prevents the blueprint from being truly portable. We're looking at fixing this issue.*
+
+
+## Wide Area Cassandra Cluster
+
+For critical enterprise use-cases, you'll want to run your Cassandra cluster across multiple regions, 
+or better yet across multiple cloud providers. This gives the highest level of availability for 
+the service.
+
+Try running:
+
+{% highlight bash %}
+% ${BROOKLYN_HOME}/bin/brooklyn launch --app brooklyn.demo.WideAreaCassandraCluster \
+  --location "aws-ec2:us-east-1,aws-ec2:us-west-2"
+{% endhighlight %}
+
+This launches the class ``WideAreaCassandraCluster`` across two AWS regions.
+
+Cassandra provides some great support for this with the 
+[EC2MultiRegionSnitch](http://www.datastax.com/docs/1.1/cluster_architecture/replication)
+The 
+[snitch](http://www.datastax.com/docs/1.1/cluster_architecture/replication#snitches)
+maps IPs to racks and data centers; it defines how the nodes are grouped together within the overall 
+network topology. For wide-area deployments, it must also deal with when to use the private IPs 
+(within a region) and the public IPs (between regions).
+You'll need a more generic snitch if you're going to span different cloud providers.
+Brooklyn has a custom MultiCloudSnitch that we're looking to contribute back to Cassandra.
+
+The important piece of code in ``WideAreaCassandraCluster`` is:
+
+{% highlight java %}
+public class WideAreaCassandraCluster extends AbstractApplication {
+  public void init() {
+    addChild(EntitySpec.create(CassandraFabric.class)
+        .configure(CassandraCluster.CLUSTER_NAME, "Brooklyn")
+        .configure(CassandraCluster.INITIAL_SIZE, 2) // per location
+        .configure(CassandraCluster.ENDPOINT_SNITCH_NAME, "brooklyn.entity.nosql.cassandra.customsnitch.MultiCloudSnitch")
+        .configure(CassandraNode.CUSTOM_SNITCH_JAR_URL, "classpath://brooklyn/entity/nosql/cassandra/cassandra-multicloud-snitch.jar"));
+  }
+}
+{% endhighlight %}
+
+The code below shows the wide-area example with the high-availability policies from the previous section also configured:
+
+{% highlight java %}
+public class WideAreaCassandraCluster extends AbstractApplication {
+  public void init() {
+    addChild(EntitySpec.create(CassandraFabric.class)
+        .configure(CassandraCluster.CLUSTER_NAME, "Brooklyn")
+        .configure(CassandraCluster.INITIAL_SIZE, 2) // per location
+        .configure(CassandraCluster.ENDPOINT_SNITCH_NAME, "brooklyn.entity.nosql.cassandra.customsnitch.MultiCloudSnitch")
+        .configure(CassandraNode.CUSTOM_SNITCH_JAR_URL, "classpath://brooklyn/entity/nosql/cassandra/cassandra-multicloud-snitch.jar")
+        .configure(CassandraFabric.MEMBER_SPEC, EntitySpec.create(CassandraCluster.class)
+            .configure(CassandraCluster.MEMBER_SPEC, EntitySpec.create(CassandraNode.class)
+                .policy(PolicySpec.create(ServiceFailureDetector.class))
+                .policy(PolicySpec.create(ServiceRestarter.class)
+                    .configure(ServiceRestarter.FAILURE_SENSOR_TO_MONITOR, ServiceFailureDetector.ENTITY_FAILED)))
+            .policy(PolicySpec.create(ServiceReplacer.class)
+                 .configure(ServiceReplacer.FAILURE_SENSOR_TO_MONITOR, ServiceRestarter.ENTITY_RESTART_FAILED))));
+  }
+}
+{% endhighlight %}
+
+To run Cassandra across multiple clouds, try running:
+
+{% highlight bash %}
+% ${BROOKLYN_HOME}/bin/brooklyn launch --app brooklyn.demo.WideAreaCassandraCluster \
+  --location "aws-ec2:us-east-1,google-compute-engine,rackspace-cloudservers-uk"
+{% endhighlight %}
+
+
+### Testing your Wide-Area Cluster
+
+You can again use the ``cassandra-stress`` command line tool to test the wide-area cluster.
+
+Note that the replication strategy (such as 
+[NetworkTopologyStrategy](http://www.datastax.com/docs/1.0/cluster_architecture/replication#networktopologystrategy)
+is specified when creating a 
+[keyspace](http://www.datastax.com/documentation/cassandra/1.2/webhelp/index.html#cassandra/configuration/configStorage_r.html).
+The example below specifies a minimum of 1 replica in each datacenter.
+
+To do updates against a node in a given availability zone:
+
+{% highlight bash %}
+NODE_IDS=<your node hostname>
+/tmp/brooklyn-aled/installs/CassandraNode/1.2.9/apache-cassandra-1.2.9/tools/bin/cassandra-stress \
+    --nodes ${NODE_IDS} \
+    --replication-strategy NetworkTopologyStrategy \
+    --strategy-properties=us-east-1:1,us-west-2:1 \
+    --progress-interval 1 \
+    --num-keys 10000 \
+    --operation INSERT
+{% endhighlight %}
+
+To check that the same data is available from a different region, target the reads
+against an appropriate node:
+
+{% highlight bash %}
+NODE_IDS=<your node hostname>
+/tmp/brooklyn-aled/installs/CassandraNode/1.2.9/apache-cassandra-1.2.9/tools/bin/cassandra-stress \
+    --nodes ${NODE_IDS} \
+    --replication-strategy NetworkTopologyStrategy \
+    --strategy-properties=us-east-1:1,us-west-2:1 \
+    --progress-interval 1 \
+    --num-keys 10000 \
+    --operation READ
+{% endhighlight %}
+
+To really test this, you may want to simulate the failure of a region first.
+You can kill the VMs or ``kill -9`` the processes. But remember that if Brooklyn policies are configured
+they will by default restart the processes automatically! You can disable the Brooklyn policies through 
+the brooklyn web-console (select the entity, go the policies tab, select the policy, and click "disable").
+
+
+## Putting it all together: CumulusRDF
+
+If you want to try this with a real example application using the Cassandra cluster, take a look at
+[CumulusRDF](https://code.google.com/p/cumulusrdf). There is an example Brooklyn application at:
+
+{% highlight bash %}
+% ${BROOKLYN_HOME}/bin/brooklyn launch --app brooklyn.demo.CumulusRDFApplication \
+  --location "aws-ec2:us-east-1"
+{% endhighlight %}
+
+
+## Contact us!
+
+If you encounter any difficulties or have any comments, please [tell us]({{ site.path.guide }}/meta/contact.html) and we'll do our best to help.

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/201818b0/docs/guide/use/examples/nosql-cassandra/index.md
----------------------------------------------------------------------
diff --git a/docs/guide/use/examples/nosql-cassandra/index.md b/docs/guide/use/examples/nosql-cassandra/index.md
new file mode 100644
index 0000000..88c1dce
--- /dev/null
+++ b/docs/guide/use/examples/nosql-cassandra/index.md
@@ -0,0 +1,7 @@
+---
+layout: guide-normal
+title: Cassandra Clusters
+toc: /guide/toc.json
+---
+
+{% readj cassandra.include.md %}

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/201818b0/docs/guide/use/examples/simple-web-cluster.md
----------------------------------------------------------------------
diff --git a/docs/guide/use/examples/simple-web-cluster.md b/docs/guide/use/examples/simple-web-cluster.md
new file mode 100644
index 0000000..cd2b659
--- /dev/null
+++ b/docs/guide/use/examples/simple-web-cluster.md
@@ -0,0 +1,9 @@
+---
+layout: guide-normal
+title: Elastic Web Cluster
+toc: /guide/toc.json
+---
+
+<!-- file kept to preserve old links; remove when link not used -->
+
+{% readj webcluster/webcluster.include.md %}

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/201818b0/docs/guide/use/examples/toc.json
----------------------------------------------------------------------
diff --git a/docs/guide/use/examples/toc.json b/docs/guide/use/examples/toc.json
new file mode 100644
index 0000000..4bca3a1
--- /dev/null
+++ b/docs/guide/use/examples/toc.json
@@ -0,0 +1,13 @@
+[
+{ "title": "Elastic Web Cluster",
+  "file":  "{{ site.path.guide }}/use/examples/webcluster/index.html" },
+{ "title": "Global Web Fabric",
+  "file":  "{{ site.path.guide }}/use/examples/global-web-fabric/index.html" },
+{ "title": "Whirr Hadoop Cluster",
+  "file":  "{{ site.path.guide }}/use/examples/whirrhadoop/index.html" },
+{ "title": "Publish-Subscribe Messaging",
+  "file":  "{{ site.path.guide }}/use/examples/messaging/index.html" },
+{ "title": "Cassandra Cluster",
+  "file":  "{{ site.path.guide }}/use/examples/nosql-cassandra/index.html" }
+
+]

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/201818b0/docs/guide/use/examples/webcluster.md
----------------------------------------------------------------------
diff --git a/docs/guide/use/examples/webcluster.md b/docs/guide/use/examples/webcluster.md
new file mode 100644
index 0000000..cd2b659
--- /dev/null
+++ b/docs/guide/use/examples/webcluster.md
@@ -0,0 +1,9 @@
+---
+layout: guide-normal
+title: Elastic Web Cluster
+toc: /guide/toc.json
+---
+
+<!-- file kept to preserve old links; remove when link not used -->
+
+{% readj webcluster/webcluster.include.md %}

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/201818b0/docs/guide/use/examples/webcluster/index.md
----------------------------------------------------------------------
diff --git a/docs/guide/use/examples/webcluster/index.md b/docs/guide/use/examples/webcluster/index.md
new file mode 100644
index 0000000..1fd80dd
--- /dev/null
+++ b/docs/guide/use/examples/webcluster/index.md
@@ -0,0 +1,7 @@
+---
+layout: guide-normal
+title: Elastic Web Cluster
+toc: /guide/toc.json
+---
+
+{% readj webcluster.include.md %}

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/201818b0/docs/guide/use/examples/webcluster/webcluster.include.md
----------------------------------------------------------------------
diff --git a/docs/guide/use/examples/webcluster/webcluster.include.md b/docs/guide/use/examples/webcluster/webcluster.include.md
new file mode 100644
index 0000000..e135998
--- /dev/null
+++ b/docs/guide/use/examples/webcluster/webcluster.include.md
@@ -0,0 +1,124 @@
+      
+{% readj ../before-begin.include.md %}
+
+## Simple Web Server
+
+Go to this particular example's directory:
+
+{% highlight bash %}
+% cd simple-web-cluster
+{% endhighlight %}
+
+The CLI needs to know where to find your compiled examples. You can set this up by exporting
+the ``BROOKLYN_CLASSPATH`` environment variable in the following way:
+
+{% highlight bash %}
+% export BROOKLYN_CLASSPATH=$(pwd)/target/classes
+{% endhighlight %}
+
+The project ``simple-web-cluster`` includes several deployment descriptors
+for rolling out a web application, under ``src/main/java``.
+
+
+
+The simplest of these, ``SingleWebServerExample``, starts JBoss on a single machine with a "Hello World" war deployed,
+with a single line:
+
+{% highlight java %}
+public class SingleWebServerExample extends AbstractApplication {
+    private static final String WAR_PATH = "classpath://hello-world-webapp.war";
+
+    @Override
+    public void init() {
+        addChild(EntitySpec.create(JBoss7Server.class)
+                .configure("war", WAR_PATH)
+                .configure("httpPort", 8080));
+    }
+}
+{% endhighlight %}
+
+You can run this as follows (on *nix or Mac, assuming `ssh localhost` requires no password or passphrase):
+
+{% highlight bash %}
+% ${BROOKLYN_HOME}/bin/brooklyn launch --app brooklyn.demo.SingleWebServerExample \
+  --location localhost
+{% endhighlight %}
+
+
+Then visit the webapp on port 8080, or the Brooklyn console on localhost:8081.
+Note that the installation may take some time, because the default deployment downloads the software from
+the official repos.  You can monitor start-up activity for each entity in the ``Activity`` pane in the management console,
+and see more detail by tailing the log file (``tail -f brooklyn.log``).
+
+With appropriate setup (as described [here]({{ site.path.guide }}/use/guide/management/index.html#startup-config)) 
+this can also be deployed to your favourite cloud, let's pretend it's Amazon Ireland, as follows: 
+
+{% highlight bash %}
+% ${BROOKLYN_HOME}/bin/brooklyn launch --app brooklyn.demo.SingleWebServerExample \
+  --location aws-ec2:eu-west-1
+{% endhighlight %}
+
+
+## Elastic Three-Tier
+
+Ready for something more interesting?  Try this:
+
+{% highlight bash %}
+% ${BROOKLYN_HOME}/bin/brooklyn launch --app brooklyn.demo.WebClusterDatabaseExample \
+  --location localhost
+{% endhighlight %}
+
+This launches the class ``WebClusterDatabaseExample`` (also described in the [walkthrough]({{ site.path.guide }}/start/walkthrough/index.html))
+which launches a pool of web-servers -- of size 1 initially,
+but manually configurable (if you stop the policy first, in the GUI, then use the ``resize`` effector) --
+with an Nginx load-balancer set up in front of them, and backed by a MySQL database.
+
+The essential code fragment looks like this:
+
+{% highlight java %}
+public class WebClusterDatabaseExample extends AbstractApplication {
+    public static final String WAR_PATH = "classpath://hello-world-sql-webapp.war";
+    
+    public static final String DB_SETUP_SQL_URL = "classpath://visitors-creation-script.sql";
+    
+    public static final String DB_TABLE = "visitors";
+    public static final String DB_USERNAME = "brooklyn";
+    public static final String DB_PASSWORD = "br00k11n";
+
+    @Override
+    public void init() {
+        MySqlNode mysql = addChild(EntitySpec.create(MySqlNode.class)
+                .configure("creationScriptUrl", DB_SETUP_SQL_URL));
+        
+        ControlledDynamicWebAppCluster web = addChild(EntitySpec.create(ControlledDynamicWebAppCluster.class)
+                .configure("memberSpec", EntitySpec.create(JBoss7Server.class)
+                        .configure("httpPort", "8080+")
+                        .configure("war", WAR_PATH)
+                        .configure(javaSysProp("brooklyn.example.db.url"), 
+                                formatString("jdbc:%s%s?user=%s\\&password=%s", 
+                                        attributeWhenReady(mysql, MySqlNode.MYSQL_URL), DB_TABLE, DB_USERNAME, DB_PASSWORD))));
+        
+        web.getCluster().addPolicy(AutoScalerPolicy.builder().
+                        metric(DynamicWebAppCluster.AVERAGE_REQUESTS_PER_SECOND).
+                        sizeRange(1, 5).
+                        metricRange(10, 100).
+                        build());
+    }
+}
+{% endhighlight %}
+
+You can, of course, try this with your favourite cloud, 
+tweak the database start script, or drop in your favourite WAR.
+
+
+## A Few Other Things
+
+The project includes variants of the examples shown here, 
+including alternative syntax (the `*Alt*` files), 
+and a web-only cluster (no database) in `WebClusterExample``.
+
+The webapp that is used is included under ``examples/hello-world-webapp``.
+
+You may wish to check out the [Global Web Fabric example]({{ site.path.guide }}/use/examples/global-web-fabric/) next.
+
+If you encounter any difficulties, please [tell us]({{ site.path.guide }}/meta/contact.html) and we'll do our best to help.

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/201818b0/docs/guide/use/guide/defining-applications/Chef.png
----------------------------------------------------------------------
diff --git a/docs/guide/use/guide/defining-applications/Chef.png b/docs/guide/use/guide/defining-applications/Chef.png
new file mode 100644
index 0000000..d899de2
Binary files /dev/null and b/docs/guide/use/guide/defining-applications/Chef.png differ

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/201818b0/docs/guide/use/guide/defining-applications/advanced-concepts.md
----------------------------------------------------------------------
diff --git a/docs/guide/use/guide/defining-applications/advanced-concepts.md b/docs/guide/use/guide/defining-applications/advanced-concepts.md
new file mode 100644
index 0000000..25b73ca
--- /dev/null
+++ b/docs/guide/use/guide/defining-applications/advanced-concepts.md
@@ -0,0 +1,137 @@
+---
+title: Advanced Concepts
+layout: guide-normal
+toc: ../guide_toc.json
+categories: [use, guide, defining-applications]
+---
+
+Lifecycle and ManagementContext
+-------------------------------
+
+Under-the-covers, at heart of the brooklyn management plane is the ``ManagementContext``. 
+This is started automatically when using launching an application using the brooklyn CLI. For programmatic use, see 
+``BrooklynLauncher.newLauncher().launch()``.
+
+A Brooklyn deployment consists of many entities in a hierarchical tree, with the privileged *application* entity at the top level.
+
+An application entity (``Application`` class) is responsible for starting the deployment of all its child entities (i.e. the entire entity tree under its ownership).
+
+An ``Application``'s ``start()`` method begins provisioning the child entities of the application (and their entities, recursively). 
+
+Provisioning of entities typically happens in parallel automatically,
+although this can be customized. This is implemented as ***tasks*** which are tracked by the management plane and is visible in the [web-based management console]({{site.path.guide}}/use/guide/management/index.html#console).
+
+Customized provisioning can be useful where two starting entities depend on each other. For example, it is often necessary to delay start of one entity until another entity reaches a certain state, and to supply run-time information about the latter to the former.
+
+<!-- TODO ambiguous language; need a better description of the "manage" lifecycle -->
+When new entities are created, the entity is wired up to an application by giving it a parent. The entity is then explicitly "managed", which allows other entities to discover it.
+
+Typically a Brooklyn deployment has a single management context which records:
+
+*   all entities under management that are reachable by the application(s) via the parent-child relationships,
+*	the state associated with each entity,
+*	subscribers (listeners) to sensor events arising from the entities,
+*	active tasks (jobs) associated with any the entity,
+*	which Brooklyn management node is mastering (managing) each entity.
+
+<!-- TODO Distributed brooklyn not yet supported; needs clarification in docs -->
+
+In a multi-location deployment, management operates in all regions, with brooklyn entity instances being mastered in the relevant region.
+
+When management is distributed a Brooklyn deployment may consist of multiple Brooklyn management nodes each with a ``ManagementContext`` instance.
+
+<!-- TODO - Clarify the following statements.
+The management context entity forms part of the management plane. 
+The management plane is responsible for the distribution of the ``Entity`` instances across multiple machines and multiple locations, 
+tracking the transfer of events (subscriptions) between ``Entity`` instances, and the execution of tasks (often initiated by management policies).
+-->
+
+<a name="dependent"></a>
+Dependent Configuration
+-----------------------
+
+Under the covers Brooklyn has a sophisticated sensor event and subscription model, but conveniences around this model make it very simple to express cross-entity dependencies. Consider the example where Tomcat instances need to know the URL of a database (or a set of URLs to connect to a Monterey processing fabric, or other entities)
+
+{% highlight java %}
+setConfiguration(UsesJava.JAVA_OPTIONS, ImmutableMap.of("mysql.url", 
+	    attributeWhenReady(mysql, MySqlNode.MY_SQL_URL) ))
+{% endhighlight %}
+
+The ``attributeWhenReady(Entity, Sensor)`` call (a static method on the class ``DependentConfiguration``)
+causes the configuration value to be set when that given entity's attribue is ready. 
+In the example, ``attributeWhenReady()`` causes the JVM system property ``mysql.url`` to be set to the value of the ``MySqlNode.MY_SQL_URL`` sensor from ``mysql`` when that value is ready. As soon as the database URL is announced by the MySql entity, the configuration value will be available to the Tomcat cluster. 
+
+By default "ready" means being *set* (non-null) and, if appropriate, *non-empty* (for collections and strings) or *non-zero* (for numbers). Formally the interpretation of ready is that of "Groovy truth" defined by an ``asBoolean()`` method on the class and in the Groovy language extensions. 
+
+You can customize "readiness" by supplying a ``Predicate`` (Google common) or ``Closure`` (Groovy) in a third parameter. 
+This evaluates candidate values reported by the sensor until one is found to be ``true``. 
+For example, passing ``{ it.size()>=3 }`` as the readiness argument would require at least three management plane URLs.
+
+More information on this can be found in the javadoc for ``DependentConfiguration``,
+along with a few other methods such as ``valueWhenAttributeReady`` which allow post-processing of an attribute value.
+
+Note that if the value of ``CONFIG_KEY`` passed to ``Entity.getConfig`` is a Closure or Task (such as returned by ``attributeWhenReady``),
+the first access of ``Entity.getConfig(CONFIG_KEY)`` will block until the task completes.
+Typically this does the right thing, blocking when necessary to generate the right start-up sequence
+without the developer having to think through the order, but it can take some getting used to.
+Be careful not to request config information until really necessary (or to use non-blocking "raw" mechanisms),
+and in complicated situations be ready to attend to circular dependencies.
+The management console gives useful information for understanding what is happening and resolving the cycle.
+
+Location
+--------
+<!-- TODO, Clarify is how geographical location works.
+-->
+
+Entities can be provisioned/started in the location of your choice. Brooklyn transparently uses [jclouds](http://www.jclouds.org) to support different cloud providers and to support BYON (Bring Your Own Nodes). 
+
+The implementation of an entity (e.g. Tomcat) is agnostic about where it will be installed/started. When writing the application definition specify the location or list of possible locations (``Location`` instances) for hosting the entity.
+
+``Location`` instances represent where they run and indicate how that location (resource or service) can be accessed.
+
+For example, a ``JBoss7Server`` will usually be running in an ``SshMachineLocation``, which contains the credentials and address for sshing to the machine. A cluster of such servers may be running in a ``MachineProvisioningLocation``, capable of creating new ``SshMachineLocation`` instances as required.
+
+<!-- TODO, incorporate the following.
+
+The idea is that you could specify the location as AWS and also supply an image id. You could configure the Tomcat entity accordingly: specify the path if the image already has Tomcat installed, or specify that Tomcat must be downloaded/installed. Entities typically use _drivers_ (such as SSH-based) to install, start, and interact with their corresponding real-world instance. 
+-->
+
+Policies
+--------
+Policies perform the active management enabled by Brooklyn. Entities can have zero or more ``Policy`` instances attached to them. 
+
+Policies can subscribe to sensors from entities or run periodically, and
+when they run they can perform calculations, look up other values, and if deemed necessary invoke effectors or emit sensor values from the entity with which they are associated.
+
+Execution
+---------
+
+All processing, whether an effector invocation or a policy cycle, are tracked as ***tasks***. This allows several important capabilities:
+
+*	active and historic processing can be observed by operators
+*	the invocation context is available in the thread, to check entitlement (permissions) and maintain a
+hierarchical causal chain even when operations are run in parallel
+*	processing can be managed across multiple management nodes
+
+Some executions create new entities, which can then have tasks associated with them, and the system will record, for example, that a start efector on the new entity is a task associated with that entity, with that task
+created by a task associated with a different entity.
+
+The execution of a typical overall start-up sequence is shown below:
+
+[![Brooklyn Flow Diagram](brooklyn-flow-websequencediagrams.com-w400.png "Brooklyn Flow Diagram" )](brooklyn-flow-websequencediagrams.com.png)
+
+
+## Integration
+
+One vital aspect of Brooklyn is its ability to communicate with the systems it starts. This is abstracted using a ***driver*** facility in Brooklyn, where a
+driver describes how a process or service can be installed and managed using a particular technology.
+
+For example, a ``TomcatServer`` may implement start and other effectors using a ``TomcatSshDriver`` which inherits from ``JavaSoftwareProcessSshDriver`` (for JVM and JMX start confguration), inheriting from ``AbstractSoftwareProcessSshDriver``
+(for SSH scripting support).
+
+Particularly for sensors, some technologies are used so frequently that they are
+packaged as ***feeds*** which can discover their configuration (including from drivers). These include JMX and HTTP (see ``JmxFeed`` and ``HttpFeed``).
+
+Brooklyn comes with entity implementations for a growing number of commonly used systems, including various web application servers, databases and NoSQL data stores, and messaging systems. See: [Extras]({{site.path.guide}}/use/guide/extras/index.html).
+
+

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/201818b0/docs/guide/use/guide/defining-applications/archetype.md
----------------------------------------------------------------------
diff --git a/docs/guide/use/guide/defining-applications/archetype.md b/docs/guide/use/guide/defining-applications/archetype.md
new file mode 100644
index 0000000..4894af7
--- /dev/null
+++ b/docs/guide/use/guide/defining-applications/archetype.md
@@ -0,0 +1,64 @@
+---
+title: Common Usage
+layout: guide-normal
+toc: ../guide_toc.json
+categories: [use, guide, defining-applications]
+---
+
+### Maven Archetype
+
+Brooklyn includes a maven archetype, which can be used to create the project structure for a new application.
+
+This can be done interactively using:
+{% highlight bash %}
+$ mvn archetype:generate
+{% endhighlight %}
+
+The user will be prompted for the archetype to use (i.e. group "io.brooklyn" 
+and artifact "brooklyn-archetype-quickstart"), as well as options for the project 
+to be created.
+
+Alternatively, all options can be supplied at the command line. For example, 
+if creating a project named "autobrick" for "com.acme":
+
+{% highlight bash %}
+$ mvn archetype:generate \
+	-DarchetypeGroupId=io.brooklyn \
+	-DarchetypeArtifactId=brooklyn-archetype-quickstart \
+	-DarchetypeVersion={{ site.brooklyn-version }} \
+	-DgroupId=com.acme -DartifactId=autobrick \
+	-Dversion=0.1.0-SNAPSHOT \
+	-DpackageName=com.acme.autobrick \
+	-DinteractiveMode=false
+{% endhighlight %}
+
+This will create a directory with the artifact name (e.g. "autobrick" in the example above).
+Note that if run from a directory containing a pom, it will also modify that pom to add this as a module!
+
+The project will contain an example app. You can run this, and also replace it with your own
+application code.
+
+To build, run the commands:
+
+{% highlight bash %}
+$ cd autobrick
+$ mvn clean install assembly:assembly
+{% endhighlight %}
+
+The assembly command will build a complete standalone distribution archive in `target/autobrick-0.1.0-SNAPSHOT-dist.tar.gz`,
+suitable for redistribution and containing `./start.sh` in the root.
+
+An unpacked equivalent is placed in `target/autobrick-0.1.0-SNAPSHOT-dist`,
+thus you can run the single-node sample locally with:
+
+{% highlight bash %}
+$ cd target/autobrick-0.1.0-SNAPSHOT-dist/autobrick-0.1.0-SNAPSHOT/
+$ ./start.sh launch --single
+{% endhighlight %}
+
+This `start.sh` script has all of the same options as the default `brooklyn` script, 
+including `./start.sh help` and the `--location` argument for `launch`,
+with a couple of extra `launch` options for the sample blueprints in the archetype project:
+
+- `./start.sh launch --single` will launch a single app-server instance
+- `./start.sh launch --cluster` will launch a cluster of app-servers

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/201818b0/docs/guide/use/guide/defining-applications/basic-concepts.md
----------------------------------------------------------------------
diff --git a/docs/guide/use/guide/defining-applications/basic-concepts.md b/docs/guide/use/guide/defining-applications/basic-concepts.md
new file mode 100644
index 0000000..c9f6645
--- /dev/null
+++ b/docs/guide/use/guide/defining-applications/basic-concepts.md
@@ -0,0 +1,95 @@
+---
+title: Basic Concepts
+layout: guide-normal
+toc: ../guide_toc.json
+categories: [use, guide, defining-applications]
+---
+
+This introduces brooklyn and describes how it simplifies the deployment and management of big applications. It is
+intended for people who are using brooklyn-supported application components (such as web/app servers, data stores)
+to be able to use brooklyn to easily start their application in multiple locations with off-the-shelf management
+policies.
+
+Entities
+--------
+
+The central concept in a Brooklyn deployment is that of an ***entity***. 
+An entity represents a resource under management, either *base* entities (individual machines or software processes) 
+or logical collections of these entities.
+
+Fundamental to the processing model is the capability of entities to be the *parent* of other entities (the mechanism by which collections are formed), 
+with every entity having a single parent entity, up to the privileged top-level ***application*** entity.
+
+Entities are code, so they can be extended, overridden, and modified. Entities can have events, operations, and processing logic associated with them, and it is through this mechanism that the active management is delivered.
+
+The main responsibilities of an entity are:
+
+- Provisioning the entity in the given location or locations
+- Holding configuration and state (attributes) for the entity
+- Reporting monitoring data (sensors) about the status of the entity
+- Exposing operations (effectors) that can be performed on the entity
+- Hosting management policies and tasks related to the entity
+
+
+Application, Parent and Membership
+-------------------------------------
+
+All entities have a ***parent*** entity, which creates and manages it, with one important exception: *applications*.
+Application entities are the top-level entities created and managed externally, manually or programmatically.
+
+Applications are typically defined in Brooklyn as an ***application descriptor***. 
+This is a Java class specifying the entities which make up the application,
+by extending the class ``AbstractApplication``, and specifying how these entities should be configured and managed.
+
+All entities, including applications, can be the parent of other entities. 
+This means that the "child" is typically started, configured, and managed by the parent.
+For example, an application may be the parent of a web cluster; that cluster in turn is the parent of web server processes.
+In the management console, this is represented hierarchically in a tree view.
+
+A parallel concept is that of ***membership***: in addition to one fixed parent,
+and entity may be a ***member*** of any number of special entities called ***groups***.
+Membership of a group can be used for whatever purpose is required; 
+for example, it can be used to manage a collection of entities together for one purpose 
+(e.g. wide-area load-balancing between locations) even though they may have been
+created by different parents (e.g. a multi-tier stack within a location).
+
+
+Configuration, Sensors and Effectors
+------------------------------------
+
+### Configuration
+
+All entities contain a map of config information. This can contain arbitrary values, typically keyed under static ``ConfigKey`` fields on the ``Entity`` sub-class. These values are inherited, so setting a configuration value at the
+application level will make it available in all entities underneath unless it is overridden.
+
+Configuration is propagated when an application "goes live" (i.e. it becomes "managed", either explicitly or when its ``start()`` method is invoked), so config values must be set before this occurs. 
+
+Configuration values can be specified in a configuration file (``~/.brooklyn/brooklyn.properties``)
+to apply universally, and/or programmatically to a specific entity and its descendants 
+by calling `.configure(KEY, VALUE)` in the entity spec when creating it.
+There is also an ``entity.setConfig(KEY, VALUE)`` method.
+
+Additionally, many common configuration parameters are available as "flags" which can be supplied as Strings when constructing
+then entity, in the form
+``EntitySpec.create˙(MyEntity.class).configure("config1", "value1").configure("config2", "value2")``. 
+
+Documentation of the flags available for individual entities can normally be found in the javadocs. 
+The ``@SetFromFlag`` annotations on ``ConfigKey`` static field definitions
+in the entity's interface is the recommended mechanism for exposing configuration options.
+
+
+### Sensors and Effectors
+
+***Sensors*** (activity information and notifications) and ***effectors*** (operations that can be invoked on the entity) are defined by entities as static fields on the ``Entity`` subclass.
+
+Sensors can be updated by the entity or associated tasks, and sensors from an entity can be subscribed to by its parent or other entities to track changes in an entity's activity.
+
+Effectors can be invoked by an entity's parent remotely, and the invoker is able to track the execution of that effector. Effectors can be invoked by other entities, but use this functionality with care to prevent too many managers!
+
+An entity consists of a Java interface (used when interacting with the entity) and a Java class. For resilience. it is recommended to store 
+the entity's state in attributes (see `getAttribute(AttributeKey)``). If internal fields can be used then the data will be lost on brooklyn 
+restart, and may cause problems if the entity is to be moved to a different brooklyn management node.
+
+Next: [Advanced Concepts]({{site.path.guide}}/use/guide/defining-applications/advanced-concepts.html).
+See also: [Management > Sensors and Effectors]({{site.path.guide}}/use/guide/management/index.html#sensors-and-effectors).
+

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/201818b0/docs/guide/use/guide/defining-applications/brooklyn-flow-websequencediagrams.com-w400.png
----------------------------------------------------------------------
diff --git a/docs/guide/use/guide/defining-applications/brooklyn-flow-websequencediagrams.com-w400.png b/docs/guide/use/guide/defining-applications/brooklyn-flow-websequencediagrams.com-w400.png
new file mode 100644
index 0000000..c2b48ba
Binary files /dev/null and b/docs/guide/use/guide/defining-applications/brooklyn-flow-websequencediagrams.com-w400.png differ

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/201818b0/docs/guide/use/guide/defining-applications/brooklyn-flow-websequencediagrams.com.png
----------------------------------------------------------------------
diff --git a/docs/guide/use/guide/defining-applications/brooklyn-flow-websequencediagrams.com.png b/docs/guide/use/guide/defining-applications/brooklyn-flow-websequencediagrams.com.png
new file mode 100644
index 0000000..78eb5c8
Binary files /dev/null and b/docs/guide/use/guide/defining-applications/brooklyn-flow-websequencediagrams.com.png differ

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/201818b0/docs/guide/use/guide/defining-applications/chef-blueprints.md
----------------------------------------------------------------------
diff --git a/docs/guide/use/guide/defining-applications/chef-blueprints.md b/docs/guide/use/guide/defining-applications/chef-blueprints.md
new file mode 100644
index 0000000..4f9612a
--- /dev/null
+++ b/docs/guide/use/guide/defining-applications/chef-blueprints.md
@@ -0,0 +1,277 @@
+---
+title: Using Chef in YAML Blueprints
+layout: guide-normal
+toc: ../guide_toc.json
+categories: [use, guide, defining-applications]
+---
+
+This guide describes how Brooklyn entities can be easily created from Chef cookbooks.
+As of this writing (May 2014) some of the integration points are under active development,
+and comments are welcome.
+A plan for the full integration is online [here](https://docs.google.com/a/cloudsoftcorp.com/document/d/18ZwzmncbJgJeQjnSvMapTWg6N526cvGMz5jaqdkxMf8).  
+
+This guide assumes you are familiar with the basics of [creating YAML blueprints](creating-yaml.html).
+
+
+## What you need to know about Chef
+
+Chef works in two different modes, *server* and *solo*. *Server* is where the Chef client talks to a central server
+to retrieve information about its roles, policies and cookbooks (where a cookbook defines how to install and
+configure a particular piece of software). With *solo*, the client works in isolation, therefore its configuration
+and cookbooks must be supplied by another means.
+
+Chef *client* is the Chef agent. This is a Ruby application which is installed on each and every managed host. When
+invoked in server mode, it will contact the Chef server to check for updates to cookbooks and policy; it then "runs"
+the recipes in its run lists, to converge the machine to a known state. In solo mode, it reads the locally-maintained
+cookbooks and policies. The client may be run as a daemon that checks the server regularly, or it could merely be
+run manually when required.
+
+The *policy* is a set of rules on the Chef server. A client starts with a set of *attributes*, which could be as
+simple as its name and a recipe runlist, or which may involve a more complex set of attributes about how it is to be
+configured. The client then augments this with auto-detected metadata - a tool called `ohai` is run that collects
+detailed information about the host. Next, the policy on the server modifies these attributes - overriding some,
+setting defaults for others - to produce a final set of attributes. It is these which are the input to the recipes.
+Finally, the attributes are uploaded to the server where they are stored as metadata for the node, where they can be
+inspected and modified by the system operator.
+
+Also of interest is `knife`, which is the workstation toolkit for Chef. Typically this would be installed on the
+operation engineer's workstation, where it would be used to interact with the Chef server and clients. Of particular
+interest to us is the *bootstrap* operation, which is used for setting up new Chef clients - given a virtual machine,
+it will install the Chef client on it, configure it with enough information to find the Chef server and performs its
+first run, and then kicks off the Chef client for the first time.
+
+There is often a preconception about how a Chef client is bootstrapped; mistakenly, there is the belief that the
+`knife` tool configures the Chef server with information about the client, and the client finds out about itself from
+the server. This is not the case - the bootstrap operation does not involve `knife` talking to the server. Instead,
+`knife` packages up all of the required information and sends it to the client - the client will then introduce
+itself to the server, passing on its configuration.
+
+This diagram summarises the interaction between Brooklyn, the new node, and the various Chef tools. Note that there
+is no interaction between the AMP Server and the Chef Server.
+
+[![Chef Flow Diagram](Chef.png "Chef Flow Diagram" )](Chef.png)
+
+### How Brooklyn interacts with Chef
+
+Brooklyn understands both the *server* and *solo* modes of operation. Server mode utilises the `knife` toolkit, and
+therefore `knife` must be installed onto the AMP server and configured appropriately. Solo mode does not have any
+special requirements; when running in solo mode, Brooklyn will install and configure the Chef client over SSH, just
+like it does most other kinds of entities.
+
+
+## Creating Blueprints from Chef
+
+In a nutshell, a new Chef-based entity can be defined as a service by specifying
+`chef:cookbook_name` as the `service_type`, along with a collection of optional configuration.
+An illustrative example is below:
+
+{% highlight yaml %}
+{% readj example_yaml/mysql-chef-1.yaml %}
+{% endhighlight %}
+
+*This works without any installation: try it now, copying-and-pasting to the Brooklyn console.
+(Don't forget to add your preferred `location: some-cloud` to the spec.)*  
+
+We'll now walk through the important constituent parts,
+and then proceed to describing things which can be done to simplify the deployment.
+
+
+### Cookbook Primary Name
+
+The first thing to note is the type definition:
+
+    - type: chef:mysql
+
+This indicates that the Chef entity should be used (`brooklyn.entity.chef.ChefEntity`) 
+to interpret and pass the configuration,
+and that it should be parameterised with a `brooklyn.chef.cookbook.primary.name` of `mysql`.
+This is the cookbook namespace used by default for determining what to install and run.
+
+
+### Importing Cookbooks
+
+Next we specify which cookbooks are required and where they can be pulled from:
+
+      cookbook_urls:
+        mysql: https://github.com/opscode-cookbooks/mysql/archive/v4.0.12.tar.gz
+        openssl: https://github.com/opscode-cookbooks/openssl/archive/v1.1.0.tar.gz
+        build-essential: https://github.com/opscode-cookbooks/build-essential/archive/v1.4.4.tar.gz
+
+Here, specific versions are being downloaded from the canonical github repository.
+Any URL can be used, so long as it is resolvable on either the target machine or the
+Brooklyn server; this includes `file:` and `classpath:` URLs.
+
+The archive can be ZIP or TAR or TGZ.
+
+The structure of the archive must be that a single folder is off the root,
+and in that folder contains the usual Chef recipe and auxiliary files.
+For example, the archive might contain `mysql-master/recipes/server.rb`.
+Archives such as those above from github match this format.  
+The name of that folder does not matter, as often they contain version information.
+When deployed, these will be renamed to match the short name (the key in the `cookbooks_url` map,
+for instance `mysql` or `openssl`).
+
+If Chef server is configured (see below), this section can be omitted.
+
+
+### Launch Run List and Attributes
+
+The next part is to specify the Chef run list and attributes to store when launching the entity: 
+
+      launch_run_list:
+      - mysql::server
+      
+      launch_attributes:
+        mysql:
+          server_root_password: p4ssw0rd
+          server_repl_password: p4ssw0rd
+          server_debian_password: p4ssw0rd
+
+For the `launch_run_list`, you can use either the YAML `- recipe` syntax or the JSON `[ "recipe" ]` syntax.
+
+The `launch_attributes` key takes a map which will be stored against the `node` object in Chef.
+Thus in this example, the parameter `node['mysql']['server_root_password']` required by the mysql blueprint
+is set as specified.
+
+You can of course set many other attributes in this manner, in addition to those that are required!  
+
+
+### Simple Monitoring
+
+The final section determines how Brooklyn confirms that the service is up.
+Sophisticated solutions may install monitoring agents as part of the `launch_run_list`,
+with Brooklyn configured to read monitoring information to confirm the launch was successful.
+However for convenience, two common mechanisms are available out of the box:
+
+      #service_name: mysqld
+      pid_file: /var/run/mysqld/mysqld.pid
+
+If `service_name` is supplied, Brooklyn will check the return code of the `status` command
+run against that service, ensuring it is 0.  (Note that this is not universally reliable,
+although it is the same mechanism which Chef typically uses to test status when determining
+whether to start a service. Some services, e.g. postgres, will return 0 even if the service
+is not running.)
+
+If a `pid_file` is supplied, Brooklyn will check whether a process with the PID specified in that
+file is running. This has been selected for mysql because it appears to be more portable:
+the service name varies among OS's:  it is `mysqld` on CentOS but `mysql` on Ubuntu!
+
+
+## Making it Simpler: Writing Chef for Blueprints
+
+The example we've just seen shows how existing Chef cookbooks can be
+used as the basis for entities.  If you're *writing* the Chef recipes, 
+there are a few simple techniques we've established with the Chef community
+which make blueprints literally as simple as:
+
+    - type: chef:mysql
+      brooklyn.config:
+        mysql_password: p4ssw0rd
+        pid_file: /var/run/mysqld/mysqld.pid
+
+
+### Some Basic Conventions
+
+* **A `start` recipe**:
+  The first step is to provide a `start` recipe in `recipes/start.rb`;
+  if no `launch_run_list` is supplied, this is what will be invoked to launch the entity.
+  It can be as simple as a one-line file:
+
+      include_recipe 'mysql::server'
+
+* **Using `brooklyn.config`**:
+  All the `brooklyn.config` is passed to Chef as node attributes in the `node['brooklyn']['config']` namespace.
+  Thus if the required attributes in the mysql recipe are set to take a value set in
+  `node['brooklyn']['config']['mysql_password']`, you can dispense with the `launch_attributes` section.
+
+
+## Using Chef Server
+
+The examples so far have not required Chef Server, so they will work without any external
+Chef dependencies (besides the built-in install from `https://www.opscode.com/chef/install.sh`
+and the explicitly referenced cookbooks).  If you use Chef Server, however, you'll want your
+managed nodes to be integrated with it.  This is easy to set up, with a few options:
+
+If you have `knife` set up in your shell environment, the Brooklyn Chef support will use it
+by default. If the recipes are installed in your Chef server, you can go ahead and remove
+the `cookbooks_url` section!
+
+Use of `solo` or `knife` can be forced by setting the `chef_mode` flag (`brooklyn.chef.mode` config key)
+to either of those values.  (It defaults to `autodetect`, which will use `knife` if it is on the path and satisfies
+sanity checks).
+
+If you want to specify a different configuration, there are a number of config keys you can use:
+
+* `brooklyn.chef.knife.executableFile`: this should be point to the knife binary to use
+* `brooklyn.chef.knife.configFile`: this should point to the knife configuration to use
+* `brooklyn.chef.knife.setupCommands`: an optional set of commands to run prior to invoking knife,
+  for example to run `rvm` to get the right ruby version on the Brooklyn server
+
+If you're interested in seeing the Chef REST API be supported directly (without knife),
+please let us know.  We'd like to see this too, and we'll help you along the way!
+ 
+
+## Tips and Tricks
+
+To help you on your way writing Chef blueprints, here are a handful of pointers
+particularly useful in this context:
+
+* Configuration keys can be inherited from the top-level and accessed using `$brooklyn:component('id').config('key_name')`.
+  An example of this is shown in the `mysql-chef.yaml` sample recipe contained in the Brooklyn code base
+  and [here](example_yaml/mysql-chef-2.yaml) for convenience.
+  Here, `p4ssw0rd` is specified only once and then used for all the attributes required by the stock mysql cookbook.  
+
+* Github tarball downloads! You'll have noticed these in the example already, but they are so useful we thought
+  we'd call them out again. Except when you're developing, we recommend using specific tagged versions rather than master.
+
+* The usual machine `provisioning.properties` are supported with Chef blueprints, 
+  so you can set things like `minRam` and `osFamily`
+
+* To see more configuration options, and understand the ones presented here in more detail, see the javadoc or
+  the code for the class `ChefConfig` in the Brooklyn code base.
+
+
+## Advanced Chef Integration
+
+### Adding Sensors and Effectors
+
+Custom sensors and effectors can be added using an `entity.initializer` section in the YAML blueprint.
+
+One common pattern is to have sensors which extract information from Ohai.
+Another common pattern is to install a monitoring agent as part of the run list,
+configured to talk to a monitoring store, and then to add a sensor feed which reads data from that store.
+
+On the effector side, you can add SSH-based effectors in the usual way.
+You can also describe additional chef converge targets following the pattern set down in
+`ChefLifecycleEffectorTasks`, making use of conveniences in `ChefSoloTasks` and `ChefServerTasks`,
+or provide effectors which invoke network API's of the systems under management
+(for example to supply the common `executeScript` effector as on the standard `MySqlNode`). 
+   
+
+### Next Steps: Simpifying sensors and effectors, transferring files, and configuring ports
+
+The Brooklyn-Chef integration is work in progress, with a few open issues we'd still like to add.
+Much of the thinking for this is set forth in the [Google document](https://docs.google.com/a/cloudsoftcorp.com/document/d/18ZwzmncbJgJeQjnSvMapTWg6N526cvGMz5jaqdkxMf8)
+indicated earlier.  If you'd like to work with us to implement these, please let us know.
+
+
+## Reference
+
+A general schema for the supported YAML is below: 
+
+```
+- type: chef:cookbook_name
+  cookbook_urls:
+    cookbook_name: url://for/cookbook.tgz
+    dependency1: url://for/dependency1.tgz
+  launch_run_list: [ "cookbook_name::start" ]
+  launch_attributes: # map of arguments to set in the chef node
+  service_name: cookbook_service
+  pid_file: /var/run/cookbook.pid
+```
+
+If you are interested in exploring the Java code for creating blueprints,
+start with the `TypedToyMySqlEntiyChef` class, which essentially does what this tutorial has shown;
+and then move on to the `DynamicToyMySqlEntiyChef` which starts to look at more sophisticated constructs.
+(Familiarity with BASH and basic Java blueprints may be useful at that stage.)
+


Mime
View raw message