brooklyn-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From henev...@apache.org
Subject [10/13] incubator-brooklyn git commit: wholesale replacement by docs folder from master
Date Fri, 09 Jan 2015 16:09:38 GMT
http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_extra/highlevel1.md
----------------------------------------------------------------------
diff --git a/docs/_extra/highlevel1.md b/docs/_extra/highlevel1.md
new file mode 100644
index 0000000..3dfafb4
--- /dev/null
+++ b/docs/_extra/highlevel1.md
@@ -0,0 +1,50 @@
+## What is Brooklyn?
+
+**brooklyn** is a library that simplifies application deployment and management.
+
+For **deployment**, it is designed to tie in with other tools, 
+giving single-click deploy and adding the concepts of 
+manageable clusters and fabrics:
+
+* many common software entities available out-of-the-box
+* integrates with [Apache Whirr](http://whirr.apache.org) 
+  to deploy well-known services such as Hadoop and elasticsearch
+  (or use POBS, plain-old-bash-scripts)
+* use PaaS's such as OpenShift, alongside self-built clusters, for maximum flexibility
+
+Brooklyn makes roll-out an integral part of the DevOps chain,
+as code which can be version-controlled and programmatically tested,
+and portable across many clouds or fixed IP machines,
+using [jclouds](http://jclouds.org) -- 
+or just hitting ``localhost`` for quick dev/test.
+
+Brooklyn's main emphasis is post-deployment, **managing** an application once it is live:
+management policies are an integral part of the deployment descriptor,
+and at runtime policies have access to all aspects of the deployment.
+They are aware of the deployment topology (hierarchical) and
+locations (machines, PaaSes, and jurisdictions), 
+as well as scripts, instrumentation, and operational goals and constraints.
+This means they're all set, once the application is launched, 
+to keep the application running optimally,
+based on whatever *optimally* means in that context.
+
+These deployment patterns and management policies are expressed as Java (and Groovy) classes,
+open-sourced here and giving you full control over what you want to happen.
+More importantly, however, this code can be shared, improved, and extended.
+
+We're still near the beginning of figuring this out: 
+[join us to make it better]({{site.path.guide}}/meta/contact.html).
+
+
+## To Get Started
+
+* See the [developer's walkthrough]({{site.path.guide}}/start/walkthrough/index.html) for a quick tour
+* Check out the [examples]({{site.path.guide}}/use/examples/), from a global web fabric with geo-DNS to a movable PaaS target
+* Jump in to the [user guide]({{site.path.guide}}/use/guide/) describing the 
+  [concepts]({{site.path.guide}}/use/guide/defining-applications/basic-concepts.html)
+  and including a [tutorial]({{site.path.guide}}/use/guide/quickstart/)
+* Or dive straight in to the code, either [reading]({{site.path.guide}}/dev/code/) about it
+  or [gitting](http://github.com/apache/incubator-brooklyn/) it
+
+If you like it, or if you have ideas how it could be better,
+[join the discussion]({{site.path.guide}}/meta/contact.html).

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_extra/list-of-blueprints.md
----------------------------------------------------------------------
diff --git a/docs/_extra/list-of-blueprints.md b/docs/_extra/list-of-blueprints.md
new file mode 100644
index 0000000..7136dd1
--- /dev/null
+++ b/docs/_extra/list-of-blueprints.md
@@ -0,0 +1,160 @@
+---
+title: Systems Available Out-of-the-Box
+layout: guide-normal
+toc: ../guide_toc.json
+categories: [use, guide]
+---
+
+brooklyn comes bundled with support for a large number of systems and entities.
+
+*Some entities are in an early-access state, and documentation is incomplete. Please contact the Brooklyn Project for assistance and clarification.*
+<!---
+.. TODO fix
+.. TODO name entities
+.. TODO include the fully qualified name of the entity
+-->
+
+<a name="web"></a>
+Web
+---
+
+### Clusters and Interfaces
+
+The class ``ControlledDynamicWebAppCluster`` creates a load-balanced cluster of web servers.
+It defaults to Nginx and JBoss 7, but this is configurable with the ``controller`` or ``controllerSpec``, and 
+the ``memberSpec`` configuration options.
+
+Most web app server processes, and some clusters and PaaS implementations,
+support the interface ``WebAppService`` which defines many sensors including requests per second.
+This allows app server metrics to interoperable across implementations in many cases.
+
+
+### JBoss Application Server
+
+Brooklyn supports JBoss 7 in the calss ``JBoss7Server``, with a wide range of
+monitoring.
+
+JBoss 6 is also supported using the different class ``JBoss6Server``.
+(The different implementation is needed due to major differences between 6 and 7,
+including switching from JMX to HTTP/JSON as the preferred metrics mechanism.)
+
+
+### Apache Tomcat
+
+Apache Tomcat is supported in the class ``TomcatServer``.
+(Note that this currently uses a legacy Brooklyn class hierarchy,
+and could benefit from being ported to the ``JavaSoftwareProcessSshDriver`` implementation.)
+
+
+### Nginx Load Balancer
+
+Nginx provides clustering support for several web/app servers.
+
+The install process downloads the sources for both the service and the sticky session module, configures them using GNI
+autoconf and compiles them. This requires gcc and autoconf to be installed. The install script also uses the yum package manager (if available) to install openssl-devel which is required to build the service. This will only work on RHEL or CentOS Linux systems, but the install process should proceed on a vanilla system with development tools available.
+
+On debian/ubuntu to build nginx you can get the required libraries with: 
+``apt-get install zlib1g-dev libdigest-sha-perl libssl-dev``.
+(The entity install script will attempt to do this with sudo, 
+but that may fail if sudo access is not available.) 
+
+
+<a name="database"></a>
+Database
+--------
+
+### MySQL
+
+MySQL is one of the most popular relational databases.
+Brooklyn supports setting up individual MySQL nodes with arbitrary configuration,
+which may be used to create multiple nodes using back-up and synchronization processes as desired.
+(If certain patterns for configuring multiple nodes become popular, these could be
+added as Brooklyn entities.)  
+
+
+### Apache Derby
+
+*This entity is in the sandbox.* 
+
+Brooklyn supports Apache Derby, a pure-Java SQL database. For setting up an instance of a server see ``DerbySetup``.
+
+
+<a name="nosql"></a>
+NoSQL
+-----
+
+*The NoSQL entities may not be complete.* 
+
+### Redis
+
+Redis is a distributed key-value store, supporting master/slave replication of a store as a clustered cache. This gives
+a series of read-only slaves and a single read-write master, which propagates to the slaves with eventual consistency.
+
+
+### MongoDB
+
+
+### Cassandra
+
+
+### CouchBase
+
+
+<a name="messaging"></a>
+Messaging
+---------
+
+### Qpid
+
+
+Qpid support provides a JMS broker, running over AMQP. This exposes JMS queues and topics as entities as well.
+See ``QpidSetup`` for instantiating a broker.
+
+### ActiveMQ
+
+
+ActiveMQ support provides a JMS broker. This exposes JMS queues and topics as entities as well. See ``ActiveMQSetup`` for
+instantiating a broker.
+
+### RabbitMQ
+
+
+<a name="downstream-projects"></a>
+Downstream Projects
+-------------------
+
+Downstream projects include those below.
+
+### Apache Whirr
+
+https://github.com/brooklyncentral/brooklyn-whirr
+
+Whirr allows running a variety of services on cloud providers and on localhost. This is done by providing a ``recipe`` which describes what services to launch. You can find an example of how Brooklyn integrates with Whirr [here](/use/examples/whirrhadoop/index.html#custom-whirr-recipe).
+
+### OpenShift
+
+https://github.com/cloudsoft/brooklyn-openshift
+
+### CloudFoundry
+
+https://github.com/cloudsoft/brooklyn-cloudfoundry and https://github.com/cloudsoft/brooklyn-cloudfoundry
+
+### MPI
+
+https://github.com/cloudsoft/brooklyn-openmpi
+
+### Waratek
+
+https://github.com/cloudsoft/brooklyn-waratek
+
+### MapR
+
+https://github.com/cloudsoft/brooklyn-mapr
+
+### Cloudera CDH
+
+https://github.com/cloudsoft/brooklyn-cdh
+
+### Drupal and Wordpress
+
+https://github.com/cloudsoft/brooklyn-social-apps

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_extra/local-artifact-repo.md
----------------------------------------------------------------------
diff --git a/docs/_extra/local-artifact-repo.md b/docs/_extra/local-artifact-repo.md
new file mode 100644
index 0000000..455a545
--- /dev/null
+++ b/docs/_extra/local-artifact-repo.md
@@ -0,0 +1,32 @@
+---
+layout: guide-normal
+title: Prepopulating a Local Artifact Repository
+toc: /guide/toc.json
+---
+
+On occasion it can be useful to have/control/prepopulate a local repository of entity installers <small>[1]</small>.
+
+The following command (run from `~/`) may be used to sync Cloudsoft's fallback repository to the local `~/.brooklyn/repository/` folder:
+
+	wget --directory-prefix=".brooklyn/repository/" --no-parent --relative --no-host-directories --reject="index.html*" --cut-dirs=2 --recursive -e robots=off --user-agent="Brooklyn Repository Sync" http://downloads.cloudsoftcorp.com/brooklyn/repository/
+
+Brooklyn's default search behaviour for installation artifacts is as follows:
+
+1.  The local `~/.brooklyn/repository/` folder.
+2.	The entity's installer's public download url (or an overridden url if one has been specified).
+3.	Cloudsoft's fallback repository.
+
+Cloudsoft's fallback repository <small>[2]</small> contains many of the installation artifacts used by current Brooklyn entities. 
+
+It is intended to prevent problems occurring when the public url for an installer changes (e.g. when several new versions of MySQL have been released). It is provided on an as-is and as-available basis.
+
+If you use this command to create a local repository, please respect the `--user-agent`. In future this will allow Cloudsoft to easily filter repository syncing behaviour from  fallback behaviour, allowing out-of-date entities to be more easily identified and updated. 
+
+<br />
+<small>
+<ol>
+<li>For example, when establishing a local cache or enterprise golden source, or when developing Brooklyn while offline, in planes, trains and automobiles, or other such situations of exemplary derring-do <small>[3]</small>.</li> 
+<li><a href="http://downloads.cloudsoftcorp.com/brooklyn/repository/">downloads.cloudsoftcorp.com/brooklyn/repository</a></li>
+<li>This one time, Cloudsoft ran a team hackathon in a castle in the remote Highlands of Scotland. Remote Highlands != reliable big pipe internet.</li>
+</ol>
+</small>

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_extra/meta/brooklyn-gpg-public-key.asc
----------------------------------------------------------------------
diff --git a/docs/_extra/meta/brooklyn-gpg-public-key.asc b/docs/_extra/meta/brooklyn-gpg-public-key.asc
new file mode 100644
index 0000000..3b515a4
--- /dev/null
+++ b/docs/_extra/meta/brooklyn-gpg-public-key.asc
@@ -0,0 +1,21 @@
+-----BEGIN PGP PUBLIC KEY BLOCK-----
+Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
+Comment: GPGTools - http://gpgtools.org
+
+mQENBFDsSLEBCAC2JxQHeXpL3oGN2IickcG9C49gkxIsws4hpasQModVipezrQi0
+9pLq4lkB01GgC2sfPH+XXE8rCpA9EL0e4wVA7JICz5AsLZAAJH91tKksL20tLMeU
+Yrbufaq1ga7ifk3JWhF4iwvkDMBKyCjrF173nI+2TwX2XfNTQpzoQGOL1bNvS4NZ
+AD9JeXGW2D996zHdSK+x3wVdY3cDECvVMuw61+5ytZrGNnyvaaWTl3lJUyydPXHQ
+5TXVtbQH5WgYCLPr4E95axJ0BoY8H+fEaG1Uax1a+xLumVWhiWNp7rMvmgcZXuJO
+fx+wXAIbRNlAHoJcdZ4NCReRxDIBQ+2HsU1zABEBAAG0bUJyb29rbHluIFByb2pl
+Y3QgKGJyb29rbHluLmlvKSAoS2V5IHVzZWQgdG8gYXV0aGVudGljYXRlIEJyb29r
+bHluIGFydGlmYWN0cykgPGJyb29rbHluLWRldkBncm91cHMuZ29vZ2xlLmNvbT6J
+AT8EEwECACkFAlDsSLECGy8FCQeGH4AHCwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIX
+gAAKCRANhinnSRLCsOdIB/4tUVShup2NHXJ9acCah8TuEN4GmN9dBiD9YsGW66SR
+/ptY0Gn9XExl2wbmQW+7TQg3QUGv8uffwYLtnMwnmCp/WwgE+uSnRmcENxa9GuTu
+PLlURKKGK0C9ljTAHwXtPcIYxPNN3BT4VB56ME1DTBRCgEvudaNSANs8/kT88kE2
+eMC7x0Uo3/P38Ob8XSOfR8c6G6nSz6jILcRBXZTPNNK4svyqF5XHIru65d3/0+mr
+bpfcDLcUQYms0MpPmO1RCHLZWwJLsPUIxNwGGnKJc8/RNEvQinK+Ap0cf+PGUQSX
+PhB6Z81ROFIVToEVZslgSiL+u4Tc7zXDfDQDY4HeLY2t
+=w/CG
+-----END PGP PUBLIC KEY BLOCK-----

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_extra/meta/contact.include.md
----------------------------------------------------------------------
diff --git a/docs/_extra/meta/contact.include.md b/docs/_extra/meta/contact.include.md
new file mode 100644
index 0000000..e0d1e4b
--- /dev/null
+++ b/docs/_extra/meta/contact.include.md
@@ -0,0 +1,9 @@
+
+Contact us on the [brooklyn mailing list](https://mail-archives.apache.org/mod_mbox/incubator-brooklyn-dev/).
+
+Additionally look for us on:
+
+* **IRC**: #brooklyncentral ([IRC??](/meta/irc.html))
+* **Twitter**: @brooklyncentral #brooklyncentral
+
+We'd love to hear from you!

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_extra/meta/contact.md
----------------------------------------------------------------------
diff --git a/docs/_extra/meta/contact.md b/docs/_extra/meta/contact.md
new file mode 100644
index 0000000..c318a1f
--- /dev/null
+++ b/docs/_extra/meta/contact.md
@@ -0,0 +1,7 @@
+---
+layout: guide-normal
+title: Contact
+toc: /guide/toc.json
+---
+
+{% readj contact.include.md %}

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_extra/meta/irc.md
----------------------------------------------------------------------
diff --git a/docs/_extra/meta/irc.md b/docs/_extra/meta/irc.md
new file mode 100644
index 0000000..9324bd2
--- /dev/null
+++ b/docs/_extra/meta/irc.md
@@ -0,0 +1,31 @@
+---
+layout: guide-normal
+title: IRC
+toc: /guide/toc.json
+---
+
+IRC is a basic group chat system used by many open source projects to enable developers to 
+collaborate and to help users communicate and support each other. Brooklyn has an IRC 
+channel where users and developers hang out.
+
+## Getting Help On IRC
+
+When you connect to the Brooklyn IRC channel, say Hi and then ask your question. It may 
+take a little while for someone to see and respond to your question. (Be 
+patient if you don't get an instant answer - unless there's already a conversation 
+happening folk will probably not have their eyes on the channel.) 
+
+You may want to do other things while you wait, like reviewing the [brooklyn mailing list](https://mail-archives.apache.org/mod_mbox/incubator-brooklyn-dev/)
+lists to see if anyone has had the same issue before.
+
+## Brooklyn IRC Channel
+
+* **IRC Server:** irc.freenode.net 
+* **Channel/Room:** #brooklyncentral
+* **Port:** 6667 (Default)
+
+If you are new to IRC, there is a web interface at [webchat.freenode.net](http://webchat.freenode.net/?channels=brooklyncentral) 
+but you may prefer to checkout the [Wikipedia IRC article](http://en.wikipedia.org/wiki/IRC#Client_software) 
+for an IRC application.
+
+You can find more information about using IRC at [irchelp.org](http://www.irchelp.org/).

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_extra/meta/toc.json
----------------------------------------------------------------------
diff --git a/docs/_extra/meta/toc.json b/docs/_extra/meta/toc.json
new file mode 100644
index 0000000..f01fa8e
--- /dev/null
+++ b/docs/_extra/meta/toc.json
@@ -0,0 +1,8 @@
+[
+{ "title": "Versions",
+  "file":  "{{ site.path.guide }}/meta/versions.html" },
+{ "title": "Verify",
+  "file":  "{{ site.path.guide }}/meta/verify.html" },
+{ "title": "Contact",
+  "file":  "{{ site.path.guide }}/meta/contact.html" }
+]

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_extra/meta/versions.md
----------------------------------------------------------------------
diff --git a/docs/_extra/meta/versions.md b/docs/_extra/meta/versions.md
new file mode 100644
index 0000000..57e64a4
--- /dev/null
+++ b/docs/_extra/meta/versions.md
@@ -0,0 +1,110 @@
+---
+layout: guide-normal
+title: Versions
+toc: /guide/toc.json
+---
+
+<!--- display which version we are using, based on where it is written -->
+
+### Brooklyn v{{ site.brooklyn-stable-version }}
+
+
+{% if site.server %} 
+> **Server (debug) mode detected.**
+
+> *Links to other versions on this page and others will likely not work when running in server/debug mode.
+Files must be copied to the brooklyncentral.github.com repo for these links to resolve correctly.*
+
+> *Debug page generated {{ site.time }}*
+{% endif %}
+
+
+{% if site.brooklyn-version contains 'SNAPSHOT' %}
+<!--- snapshot version -->
+
+  {% if site.path.guide == '' %}
+
+<!--- current version (served off root of site) is snapshot (unusual) -->
+
+This is the documentation for the current snapshot version of Brooklyn,
+generated {{ site.time | date_to_string }}.
+
+  {% else %}
+
+<!--- archive docs -->
+
+This is the documentation for a snapshot version of Brooklyn,
+generated {{ site.time | date_to_string }}.
+
+[View current documentation here.](/meta/versions.html)
+
+
+  {% endif %}
+
+NB: "Snapshot" means it is the code at a point in time,
+and that a reference to this version {{ site.brooklyn-version }}
+may resolve to different code at a different point in time.
+Where possible it is preferable to develop against a GA version
+rather than a shapshot.  
+
+{% else %}
+<!--- not snapshot -->
+
+  {% if site.path.guide == '' %}
+   
+<!--- current version (served off root of site) -->
+
+This is the documentation for the latest stable version of Brooklyn,
+generated {{ site.time | date_to_string }}.
+Other versions with documentation available are listed below.
+
+  {% else %}
+
+<!--- archive version -->
+
+This is the archived documentation for Brooklyn {{ site.brooklyn-version }}
+(generated {{ site.time }}, archived under {{ site.path.guide }}).
+
+[View current documentation here.](/meta/versions.html)
+
+  {% endif %}  
+{% endif %}
+
+
+### Version History
+
+* **[0.7.0-SNAPSHOT (master)](/v/0.7.0-SNAPSHOT)**: since 0.6.0, lots of work on yaml, persistence, policies, and more supported systems
+
+* **[0.7.0](/v/0.7.0-M1/)**: most recent milestone release
+
+* **[0.6.0](/v/0.6.0/)**: use of spec objects, chef and windows support, more clouds (Nov 2013)
+
+* **[0.5.0](/v/0.5.0/)**: includes new JS GUI and REST API, rebind/persistence support, cleaner model and naming conventions, more entities (May 2013)
+
+* **[0.4.0](/v/0.4.0/)**: initial public GA release of Brooklyn to Maven Central, supporting wide range of entities and examples (Jan 2013)
+
+Note: To prevent accidentally referring to out-of-date information,
+a banner is displayed when accessing specific versions from the archive.
+You may 
+<a href="#" onclick="set_user_versions_all();">disable all warnings</a> or
+<a href="#" onclick="clear_user_versions();">re-enable all warnings</a>.
+
+
+### Versioning
+
+Brooklyn uses the [semantic versioning](http://semver.org/) guidelines. Releases will be numbered with the following format:
+
+`Brooklyn <major>.<minor>.<patch>`
+
+Breaking backward compatibility increments the `<major>` version.
+New additions without breaking backward compatibility ups the `<minor>` version.
+Bug fixes and misc changes bumps the `<patch>` version.
+New major and minor releases zero the less significant counters.
+
+Additionally, Brooklyn's release process include Snapshots, Milestones and Release Candidates.
+
+A Snapshot (`-SNAPSHOT`) is the bleeding edge. This will not be stable.
+
+Milestone versions (`-Mn`) are frozen snapshots. Some code features may be stable, but the documentation and examples may not be complete.
+
+A Release Candidate (`-rc.n`) is a just-about-ready version. Release candidates are tested against our acceptance criteria, and qualifying builds are promoted as final.

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_extra/release.md
----------------------------------------------------------------------
diff --git a/docs/_extra/release.md b/docs/_extra/release.md
new file mode 100644
index 0000000..66f730b
--- /dev/null
+++ b/docs/_extra/release.md
@@ -0,0 +1,288 @@
+---
+layout: guide-normal
+title: Release Process
+toc: /guide/toc.json
+---
+
+**This needs substantial updating in light of the Apache move.**
+
+<!--
+TODO
+
+vote required?  see governance.
+
+
+     branch-twice-then-reversion-twice
+     e.g. from master=1.0.0_SNAPSHOT we will go to
+          create branch: v1.0.0_SNAPSHOT
+          reversion master:  1.1.0_SNAPSHOT
+          create branch and reversion:  v1.0.0_RC1, v1.0.0_SNAPSHOT
+     describe scripts for releasing
+     docs
+
+update version, using scripts
+
+push examples to repo
+
+push docs to branch and publish
+
+-->
+
+Brooklyn is published to two locations:
+
+* Sonatype, for snapshots and for staging releases
+* Maven Central, for full (GA and milestone) releases
+
+Brooklyn artifacts are generally downloaded from:
+
+1. [Maven Central](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22io.brooklyn%22), 
+   and (http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.brooklyn%22) for pre 0.7.0 versions
+2. [Apache](https://repository.apache.org/index.html#nexus-search;quick~org.apache.brooklyn),
+2. [Sonatype](https://oss.sonatype.org/index.html#nexus-search;quick~io.brooklyn) for pre 0.7.0 versions
+3. [GitHub](https://github.com/apache/incubator-brooklyn).
+
+
+To publish:
+
+* a snapshot release:
+	* mvn deploy to Sonatype
+	* (optional) publish versioned docs to brooklyncentral.github.com project
+* a (milestone) release:
+	* same as above, but with some git versioning 
+	* deploy to Sonatype, then release to Maven Central
+	* deploy a version branch to brooklyn-examples 
+	* deploy (or update) versioned docs
+* a major release:
+	* same as above, and
+	* in addition to versioned examples,  update brooklyn-examples master to match the current (stable) release
+	* in addition to versioned docs, publish full (front page) docs to brooklyncentral.github.com project
+	* bump the snapshot version in brooklyn master to the next release
+
+
+	
+## Configuration 
+
+Your .m2/settings.xml must be configured with the right credentials for Sonatype
+
+  	<servers>
+	...
+		<server>
+			<username> ... </username>
+			<password> ... </password>
+			<id>sonatype-nexus-snapshots</id>
+		</server>
+		<server>
+			<username> ... </username>
+			<password> ... </password>
+			<id>sonatype-nexus-staging</id>
+		</server>
+	...
+	</servers>
+You must be configured to sign artifacts using PGP.
+
+If this is the first time you have used Sonatype, the [Sonatype - Maven Usage Guide](https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide) is required reading.  
+
+The code snippets below use the following variables:
+{% highlight bash %}
+
+export BROOKLYN_DIR=/path/to/brooklyncentral-brooklyn
+export EXAMPLES_DIR=/path/to/brooklyncentral-brooklyn-examples
+export SITE_DIR=/path/to/brooklyncentral-brooklyncentral.github.com
+
+export SNAPSHOT_VERSION=0.6.0-SNAPSHOT
+export RELEASE_VERSION=0.6.0-M1
+{% endhighlight %}
+
+
+## Preparing a Snapshot Release
+
+### Deploy to Sonatype
+
+Execute the following:
+{% highlight bash %}
+mvn -Dbrooklyn.deployTo=sonatype -DskipTests clean install deploy
+{% endhighlight %}
+
+### (Option) Publish snapshot docs.
+
+(Only required if there have been significant changes to docs or java docs.)
+
+{% highlight bash %}
+
+cd $BROOKLYN_DIR/docs
+git checkout master
+
+if [ ! -f $SITE_DIR/index.html ] ; then echo "could not find docs in $SITE_DIR" ; exit 1 ; fi
+
+# Build the docs
+_scripts/build.sh || { echo "failed to build docs" ; exit 1 ; }
+
+# Wipe any previous edition of the same version, replacing with new build.
+rm -rf $SITE_DIR/v/$SNAPSHOT_VERSION
+mkdir $SITE_DIR/v/$SNAPSHOT_VERSION
+cp -r _site/* $SITE_DIR/v/$SNAPSHOT_VERSION/
+
+# and push, causing GitHub to republish with updated /v/$SNAPSHOT_VERSION/
+pushd $SITE_DIR
+git add -A .
+git commit -m "Updated version docs for version $SNAPSHOT_VERSION"
+git push
+popd
+
+{% endhighlight %}
+
+
+
+## Preparing a (Milestone) Release
+
+### Prepare a Release Branch
+
+{% highlight bash %}
+
+cd $BROOKLYN_DIR
+git checkout -b $RELEASE_VERSION
+usage/scripts/change-version.sh $SNAPSHOT_VERSION $RELEASE_VERSION
+git commit -a -m "Changed version to $RELEASE_VERSION"
+git push -u upstream $RELEASE_VERSION
+
+{% endhighlight %}
+
+### Deploy to Sonatype, and Close the repo.
+
+{% highlight bash %}
+mvn -Dbrooklyn.deployTo=sonatype -DskipTests clean install deploy
+{% endhighlight %}
+
+* Go to [oss.sonatype.org ... #stagingRepositories](https://oss.sonatype.org/index.html#stagingRepositories) (again, need credentials)
+* 'Close' the repo
+* Email the closed repo address to brooklyn-dev list, have people download and confirm it works.
+
+### Update the brooklyn-examples repo's version Branch
+
+{% highlight bash %}
+
+cd $EXAMPLES_DIR
+
+pushd $BROOKLYN_DIR
+git checkout $RELEASE_VERSION
+popd
+
+if [ ! -d simple-web-cluster ] ; then echo "wrong dir" ; exit 1 ; fi
+git checkout master
+git checkout -b $RELEASE_VERSION
+rm -rf *
+cp -r $BROOKLYN_DIR/examples/* .
+rm -rf target
+git add -A
+git commit -m "branch for $RELEASE_VERSION"
+git push -u origin $RELEASE_VERSION
+
+{% endhighlight %}
+
+
+### Update the Versioned Docs
+
+{% highlight bash %}
+
+cd $BROOKLYN_DIR/docs
+git checkout $RELEASE_VERSION
+
+if [ ! -f $SITE_DIR/index.html ] ; then echo "could not find docs in $SITE_DIR" ; exit 1 ; fi
+
+# Build the docs
+_scripts/build.sh || { echo "failed to build docs" ; exit 1 ; }
+
+# Wipe any previous edition of the same version, replacing with new build.
+rm -rf $SITE_DIR/v/$RELEASE_VERSION
+mkdir $SITE_DIR/v/$RELEASE_VERSION
+cp -r _site/* $SITE_DIR/v/$RELEASE_VERSION/
+
+# and push, causing GitHub to republish with updated /v/$RELEASE_VERSION/
+pushd $SITE_DIR
+git add -A .
+git commit -m "Updated version docs for version $RELEASE_VERSION"
+git push
+popd
+
+{% endhighlight %}
+	
+## Preparing a Full Release
+
+Complete *all* above steps.
+
+### Deploy to Maven Central
+
+* Confirm that the closed Sonatype repo has no errors
+* Return to [Sonatype: Staging Repositories](https://oss.sonatype.org/index.html#stagingRepositories)
+* 'Release' the repo
+
+### Deploy the Examples master branch.
+
+{% highlight bash %}
+
+cd $EXAMPLES_DIR
+
+pushd $BROOKLYN_DIR
+git checkout $RELEASE_VERSION
+popd
+
+if [ ! -d simple-web-cluster ] ; then echo "wrong dir" ; exit 1 ; fi
+git checkout master
+rm -rf *
+cp -r $BROOKLYN_DIR/examples/* .
+rm -rf target
+git add -A
+git commit -m "Updated to $RELEASE_VERSION"
+git push -u origin master
+
+{% endhighlight %}
+
+### Update the brooklyn.io Front Page Version
+
+{% highlight bash %}
+
+cd $BROOKLYN_DIR/docs
+
+pushd $SITE_DIR
+# remove old root files, but not the previous version in /v/
+if [ -f start/index.html ] ; then
+  for x in * ; do if [[ $x != "v" ]] ; then rm -rf $x ; fi ; done
+else
+  echo IN WRONG DIRECTORY $SITE_DIR - export SITE_DIR to continue
+  exit 1
+fi
+popd
+
+# re-build for hosting at / rather than at /v/VERSION/
+_scripts/build.sh --url "" || { echo "failed to build docs" ; exit 1 ; }
+
+# copy to site dir
+cp -r _site/* $SITE_DIR/
+
+# and git push
+pushd $SITE_DIR
+git add -A .
+git commit -m "Updated root docs for version $RELEASE_VERSION"
+git push
+popd
+
+{% endhighlight %}
+
+
+### Announce
+* Email the Dev and Users mailing lists.
+* Tweet from [@brooklyncentral](https://twitter.com/brooklyncentral)
+
+### Update Snapshot Version
+
+{% highlight bash %}
+
+export NEW_SNAPSHOT_VERSION=0.7.0-SNAPSHOT
+
+cd $BROOKLYN_DIR
+git checkout master
+usage/scripts/change-version.sh $SNAPSHOT_VERSION $NEW_SNAPSHOT_VERSION
+git commit -a -m "Changed version to $NEW_SNAPSHOT_VERSION"
+git push -u upstream master
+
+{% endhighlight %}

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_extra/simple_java_examples/example_files/tomcat_multi-location.java
----------------------------------------------------------------------
diff --git a/docs/_extra/simple_java_examples/example_files/tomcat_multi-location.java b/docs/_extra/simple_java_examples/example_files/tomcat_multi-location.java
new file mode 100644
index 0000000..cb92766
--- /dev/null
+++ b/docs/_extra/simple_java_examples/example_files/tomcat_multi-location.java
@@ -0,0 +1,15 @@
+// TODO Untested code; see brooklyn-example for better maintained examples!
+public class TomcatFabricApp extends AbstractApplication {
+    @Override
+    public void init() {
+        addChild(EntitySpec.create(DynamicFabric.class)
+                .configure("displayName", "WebFabric")
+                .configure("displayNamePrefix", "")
+                .configure("displayNameSuffix", " web cluster")
+                .configure("memberSpec", EntitySpec.create(ControlledDynamicWebAppCluster.class)
+                        .configure("initialSize", 2)
+                        .configure("memberSpec", : EntitySpec.create(TomcatServer.class)
+                                .configure("httpPort", "8080+")
+                                .configure("war", "/path/to/booking-mvc.war"))));
+    }
+}

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_extra/simple_java_examples/example_files/tomcat_nginx.java
----------------------------------------------------------------------
diff --git a/docs/_extra/simple_java_examples/example_files/tomcat_nginx.java b/docs/_extra/simple_java_examples/example_files/tomcat_nginx.java
new file mode 100644
index 0000000..20db33d
--- /dev/null
+++ b/docs/_extra/simple_java_examples/example_files/tomcat_nginx.java
@@ -0,0 +1,17 @@
+// TODO Untested code; see brooklyn-example for better maintained examples!
+public class TomcatClusterWithNginxApp extends AbstractApplication {
+    @Override
+    public void init() {
+        addChild(EntitySpec.create(NginxController.class)
+                .configure("domain", "brooklyn.geopaas.org")
+                .configure("port", "8000+")
+                .configure("portNumberSensor", Attributes.HTTP_PORT));
+        
+        addChild(EntitySpec.create(ControlledDynamicWebAppCluster.class)
+                .configure("controller", nginxController)
+                .configure("memberSpec", : EntitySpec.create(TomcatServer.class)
+                        .configure("httpPort", "8080+")
+                        .configure("war", "/path/to/booking-mvc.war"))
+                .configure("initialSize", 2));
+    }
+}

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_extra/simple_java_examples/example_files/tomcat_simple.java
----------------------------------------------------------------------
diff --git a/docs/_extra/simple_java_examples/example_files/tomcat_simple.java b/docs/_extra/simple_java_examples/example_files/tomcat_simple.java
new file mode 100644
index 0000000..480a333
--- /dev/null
+++ b/docs/_extra/simple_java_examples/example_files/tomcat_simple.java
@@ -0,0 +1,9 @@
+// TODO Untested code; see brooklyn-example for better maintained examples!
+public class TomcatServerApp extends AbstractApplication {
+    @Override
+    public void init() {
+        addChild(EntitySpec.create(TomcatServer.class)
+                .configure("httpPort", "8080+")
+                .configure("war", "/path/to/booking-mvc.war")));
+    }
+}

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_extra/simple_java_examples/examples.md
----------------------------------------------------------------------
diff --git a/docs/_extra/simple_java_examples/examples.md b/docs/_extra/simple_java_examples/examples.md
new file mode 100644
index 0000000..330b6d3
--- /dev/null
+++ b/docs/_extra/simple_java_examples/examples.md
@@ -0,0 +1,121 @@
+---
+title: Examples
+layout: guide-normal
+toc: ../guide_toc.json
+categories: [use, guide, defining-applications]
+---
+
+** TODO: this examples page is deprecated;
+code is out-of-date, and better examples are described on the web site.
+need to figure out if this page should be kept at all
+(indeed if the "guide" is even still relevant)**
+
+
+### Integrating with a Maven project
+
+If you have a Maven-based project, integrate this XML fragment with your pom.xml:
+
+<!-- TODO this should import from the downloads page -->
+
+{% highlight xml %}
+<dependencies>
+	<dependency>
+		<groupId>io.brooklyn</groupId>
+		<artifactId>brooklyn-all</artifactId>
+		<version>0.7.0-SNAPSHOT</version>  <!-- BROOKLYN_VERSION -->
+	</dependency>
+</dependencies>
+ 
+<repository>
+    <id>cloudsoft-releases</id>
+    <url>http://developers.cloudsoftcorp.com/download/maven2/</url>
+</repository>
+<repository>
+    <id>libs-snapshot-local</id>
+    <url>http://ccweb.cloudsoftcorp.com/maven/libs-snapshot-local/</url>
+    <snapshots>
+        <enabled>true</enabled>
+        <updatePolicy>never</updatePolicy>
+        <checksumPolicy>fail</checksumPolicy>
+    </snapshots>
+</repository>
+{% endhighlight %}
+
+
+### Starting a Tomcat Server
+
+The code below starts a Tomcat server on the local machine.
+
+The ``main`` method defines the application, and passes it to the ``BrooklynLauncher`` to be managed. 
+It is then started in a localhost location (other locations are shown in the next section).
+
+The Tomcat's configuration indicates that the given WAR should be deployed to the Tomcat server when it is started.
+
+{% highlight java %}
+{% readj example_files/tomcat_simple.java %}
+{% endhighlight %}
+
+The ``wars`` config is also supported (with config keys ``ROOT_WAR`` and ``NAMED_WARS`` the long-hand syntax);
+they accept EARs and other common archives, and can be described as files or as URLs (as Strings), 
+with URLs supporting an optional ``classpath://org/acme/resources/xxx.war`` syntax.
+
+
+### Starting Tomcat in Amazon EC2
+
+To start a tomcat node or cluster in Amazon EC2, the application is identical to that for localhost. 
+The only difference is the location supplied.
+
+The Brooklyn CLI can be used to launch the application in your choice of location, such as:
+
+{% highlight bash %}
+brooklyn launch --app TomcatServerApp --location localhost
+brooklyn launch --app TomcatServerApp --location aws-ec2:eu-west-1
+{% endhighlight %}
+
+ 
+### Starting a Tomcat Cluster with Nginx
+
+The code below starts a Tomcat cluster along with an Nginx instance, where each Tomcat server in the cluster is registered with the Nginx instance.
+
+{% highlight java %}
+{% readj example_files/tomcat_nginx.java %}
+{% endhighlight %}
+
+This creates a cluster that of Tomcat servers, along with an Nginx instance. The ``NginxController`` instance
+is notified whenever a member of the cluster joins or leaves; the entity is configured to look at the ``HTTP_PORT``
+attribute of that instance so that the Nginx configuration can be updated with the ip:port of the cluster member.
+
+<!---
+TODO things may need tidying (paragraphs, and/or eliminating any extra setConfig calls, though looks like these have gone)
+-->
+
+
+Starting a Multi-location Tomcat Fabric
+---------------------------------------
+
+<!---
+TODO this example should use several cloud providers, including Openshift, and use GeoDNS, 
+and maybe a data store and/or messaging service; it is the last "most advanced" example
+-->
+
+<!---
+FIXME Discuss above comment with Aled/Alex as it is contentious
+-->
+
+The ``ControlledDynamicWebAppCluster`` entity used above can also be used with a DynamicFabric to start
+a web-cluster in each location.
+
+{% highlight java %}
+{% readj example_files/tomcat_multi-location.java %}
+{% endhighlight %}
+
+
+Examples Source
+---------------
+
+Source code for (more up-to-date!) examples is available for download from GitHub. To retrieve the source, execute the following command:
+
+    git clone git@github.com:apache/incubator-brooklyn.git
+    cd incubator-brooklyn/examples
+
+You can also [browse the code](https://github.com/apache/incubator-brooklyn/tree/examples) on the web.

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_extra/standards.md
----------------------------------------------------------------------
diff --git a/docs/_extra/standards.md b/docs/_extra/standards.md
new file mode 100644
index 0000000..66b77b0
--- /dev/null
+++ b/docs/_extra/standards.md
@@ -0,0 +1,15 @@
+---
+layout: guide-normal
+title: Code Standards
+toc: /guide/toc.json
+---
+
+Without being too restrictive about how you have to code as part of Brooklyn,
+there are some style points that really make life easier when sharing code
+among ourselves:
+
+* Use spaces (not tabs!) with 4 spaces indentation
+* Keep line length <=128
+* Don't reformat code or organize imports unless there's very good
+  reason (this makes history and merges much harder)
+

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_extra/update-docs.md
----------------------------------------------------------------------
diff --git a/docs/_extra/update-docs.md b/docs/_extra/update-docs.md
new file mode 100644
index 0000000..ccb573e
--- /dev/null
+++ b/docs/_extra/update-docs.md
@@ -0,0 +1,14 @@
+---
+layout: guide-normal
+title: Updating the Docs
+toc: /guide/toc.json
+---
+
+<!-- TODO retire this page -->
+
+The Brooklyn docs live in the **docs** project in the Brooklyn codebase.
+It's built using standard jekyll/markdown with a few extensions.
+
+Instructions for building and working with docs are in a `README.md` file
+in that folder; for the most recent version of instructions click
+[here](https://github.com/apache/incubator-brooklyn/tree/master/docs/README.md).

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_includes/list-children.html
----------------------------------------------------------------------
diff --git a/docs/_includes/list-children.html b/docs/_includes/list-children.html
new file mode 100644
index 0000000..68f8b5c
--- /dev/null
+++ b/docs/_includes/list-children.html
@@ -0,0 +1,7 @@
+{% comment %}
+TODO style this much better
+{% endcomment %}
+
+{% for item in page.menu %}
+* [{{ item.title }}]({{ item.url }})
+{% endfor %}
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_includes/topmenu.html
----------------------------------------------------------------------
diff --git a/docs/_includes/topmenu.html b/docs/_includes/topmenu.html
index 0230f1c..c15f0df 100644
--- a/docs/_includes/topmenu.html
+++ b/docs/_includes/topmenu.html
@@ -21,32 +21,40 @@
                     <span class="icon-bar"></span>
                     <span class="icon-bar"></span>
                 </button>
-                {% if page.navgroup != 'home' %}<a class="navbar-brand" href="{{site.path.website}}/"><img src="{{site.path.style}}/img/apache-brooklyn-logo-244px-wide.png" alt="brooklyn"></a>{% endif %}
+                {% unless page.landing %}<a class="navbar-brand" href="{{site.path.website}}/"><img src="{{site.path.style}}/img/apache-brooklyn-logo-244px-wide.png" alt="brooklyn"></a>{% endunless %}
             </div>
 
             <!-- Collect the nav links, forms, and other content for toggling -->
             <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
                 <ul class="nav navbar-nav navbar-right">
-                    {% for navgroup in site.data.navgroups %}
-                    {% if navgroup.page.menu2 %}
-                    <li class="dropdown{% if page.navgroup == navgroup.id %} active{% endif %}">
-                        <a href="{{navgroup.page.url}}">{{navgroup.title}}</a>
+                  {% for item in site.data.menu %}
+                   {% if item.menu %}
+                    <li class="dropdown{% if page.breadcrumb_paths contains item.path %} active{% endif %}">
+                        <a href="{{item.url}}">{{ item.title_in_menu | downcase }}</a>
                         <ul class="dropdown-menu" role="menu">
                             <li>
-                                <a href="{{navgroup.page.url}}">{{navgroup.title_in_menu}}</a>
+                                <a href="{{item.url}}">{{item.title_in_menu}}</a>
                             </li>
                             <li class="divider"></li>
-                            {% for child in navgroup.page.menu2 %}
-                            <li>
-                                <a href="{{child.reference.url}}">{{child.reference.title}}</a>
-                            </li>
+                            {% for item2 in item.menu %}
+                              <li>
+                                {% if item2.menu_customization.dropdown_new_section %}
+                                  <div class="dropdown_new_section"><hr></div>
+                                {% endif %}
+                                <a href="{{item2.url}}">{{item2.title_in_menu}}</a>
+                                {% if item2.menu_customization.dropdown_section_header %}
+                                  <div class="dropdown_section_header"><hr></div>
+                                {% endif %}
+                              </li>
                             {% endfor %}
                         </ul>
-                    </li><!-- {{ navgroup.id }} -->
-                    {% else %}
-                    <li class="{% if page.navgroup == navgroup.id %}active{% endif %}"><a href="{{navgroup.page.url}}">{{navgroup.title}}</a></li><!-- {{ navgroup.id }} -->
-                    {% endif %}
-                    {% endfor %}
+                    </li>
+                   {% else %}
+                    <li class="{% if page.breadcrumb_paths contains item.path %}active{% endif %}">
+                      <a href="{{item.url}}">{{ item.title_in_menu | downcase }}</a>
+                    </li>
+                   {% endif %}
+                  {% endfor %}
                 </ul>
             </div><!-- /.navbar-collapse -->
             

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_layouts/guide-normal.html
----------------------------------------------------------------------
diff --git a/docs/_layouts/guide-normal.html b/docs/_layouts/guide-normal.html
index 736c838..67278e7 100644
--- a/docs/_layouts/guide-normal.html
+++ b/docs/_layouts/guide-normal.html
@@ -16,50 +16,8 @@
 # specific language governing permissions and limitations
 # under the License.
 
-layout: guide-base
+layout: website-normal
 ---
 
-{% include topmenu.html %}
-
-<div id="container">
-
-    <div id="header">
-
-        <div id="menubar">  
-
-{% include topbar.html %}                    
-
-            <form method="get" id="simple_google" class="searchform" action="http://www.google.com/search" method="get">
-                <input type="text" class="searchinput" name="brooklyn-search" placeholder=" Search: type &amp; hit enter" />
-                <input type="hidden" name="q" value="" />
-            </form>
-            
-        </div>
-                
-    </div><!--header-->
-    
-    <div id="contentcontainer">
-    
-        <div id="maincontent">
-
-{% include breadcrumbs.html %}
-
-<div id="page_notes"></div>
-
-<h1 id="header_title">{{ page.title }}</h1>
-    
 {{ content }}
 
-        </div><!--maincontent-->
-        
-      <div id="sidebar">
-      
-{% include sidebar.html %}
-
-      </div>
-
-    </div><!--contentcontainer-->
-
-</div><!--container-->
-
-{% include footer.html %}

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_layouts/guide-old.html
----------------------------------------------------------------------
diff --git a/docs/_layouts/guide-old.html b/docs/_layouts/guide-old.html
new file mode 100644
index 0000000..736c838
--- /dev/null
+++ b/docs/_layouts/guide-old.html
@@ -0,0 +1,65 @@
+---
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+# 
+#  http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+layout: guide-base
+---
+
+{% include topmenu.html %}
+
+<div id="container">
+
+    <div id="header">
+
+        <div id="menubar">  
+
+{% include topbar.html %}                    
+
+            <form method="get" id="simple_google" class="searchform" action="http://www.google.com/search" method="get">
+                <input type="text" class="searchinput" name="brooklyn-search" placeholder=" Search: type &amp; hit enter" />
+                <input type="hidden" name="q" value="" />
+            </form>
+            
+        </div>
+                
+    </div><!--header-->
+    
+    <div id="contentcontainer">
+    
+        <div id="maincontent">
+
+{% include breadcrumbs.html %}
+
+<div id="page_notes"></div>
+
+<h1 id="header_title">{{ page.title }}</h1>
+    
+{{ content }}
+
+        </div><!--maincontent-->
+        
+      <div id="sidebar">
+      
+{% include sidebar.html %}
+
+      </div>
+
+    </div><!--contentcontainer-->
+
+</div><!--container-->
+
+{% include footer.html %}

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_layouts/website-normal.html
----------------------------------------------------------------------
diff --git a/docs/_layouts/website-normal.html b/docs/_layouts/website-normal.html
index a0596e4..ba6e9b8 100644
--- a/docs/_layouts/website-normal.html
+++ b/docs/_layouts/website-normal.html
@@ -18,6 +18,8 @@
 
 # layout for most website pages, inheriting base for the menu bar, and providing a main column of content and a menu column
 # (not used for the landing page, and not used for the guide)
+#
+# menu_customizations can be used, including force_inactive to prevent highlighting
 layout: website-base
 ---
 <div class="container">
@@ -27,14 +29,60 @@ layout: website-base
             {{ content }}
         </div>
         <div class="col-md-3">
-            {% if page.menu2 %}
-            <div class="list-group">
-                <h4><a href="{{page.menu2parent.url}}" class="list-group-item{% if page.menu2parent.path == page.path %} active{% endif %}">{{ page.menu2parent.title }}</a></h4>
-                {% for child in page.menu2 %}
-                <a href="{{child.reference.url}}" class="list-group-item{% if child.reference.path == page.path %} active{% endif %}">{{ child.reference.title }}</a>
-                {% endfor %}
+            <div class="list-group side-menu">
+{% comment %}
+
+  Case 1) If we're the landing page show nothing.
+  Case 2) If we're contained in the top (landing page) menu, show our menu if we have one.
+  Case 3a) Otherwise if we're contained in a menu, show that menu.
+  Case 3b) And if we have a menu, show our menu nested.
+  
+  Note the breadcrumbs are offset:1. This is because usually breadcrumbs include the root;
+  but also if we're overriding breadcrumbs it gives us a way to set a topmenu item (making it breadcrumb[0])
+  while customizing the side menu (make self breadcrumb[1]).
+{% endcomment %}
+
+{% unless page.landing %}
+  {% if page.menu_parent %}
+    {% if page.menu_parent.landing %} {% comment %} case 2 {% endcomment %}
+              {% for item in page.breadcrumb_pages offset:1 %}
+                  <h4 class="{% if forloop.index0 > 0 %} with_preceding{% endif %}{% if forloop.rindex0 > 0 %} with_following{% endif %}">
+                    <a href="{{item.url}}" class="list-group-item{% if item.path == page.menu_path %} active{% endif %} breadcrumb breadcrumb{{ forloop.index0 }}">
+                      {{ item.title_in_menu }}</a></h4>
+              {% endfor %}
+              {% for item in page.menu %}
+                <a href="{{item.url}}" class="list-group-item{% if page.breadcrumbs contains item %} active{% endif %}">{{ item.title_in_menu }}</a>
+              {% endfor %}
+    
+    {% else %} {% comment %} case 3a {% endcomment %}
+              {% for item in page.menu_parent.breadcrumb_pages offset:1 %}
+                  <h4 class="{% if forloop.index0 > 0 %} with_preceding{% endif %}{% if forloop.rindex0 > 0 %} with_following{% endif %}">
+                    <a href="{{item.url}}" class="list-group-item{% if item.path == page.menu_path %}{% unless item.menu_customization.force_inactive %} active{% endunless %}{% endif %} breadcrumb breadcrumb{{ forloop.index0 }}">
+                      {{ item.title_in_menu }}</a></h4>
+              {% endfor %}
+              {% for item in page.menu_parent.menu %}
+                {% if (page.breadcrumb_paths contains item.menu_path) and page.menu %}
+                  {% comment %} case 3b {% endcomment %}
+                  <a href="{{item.url}}" class="list-group-item active with-sub-item">{{ item.title_in_menu }}</a>
+                  <div class="sub-item">
+                   {% for item2 in page.menu %}
+                    <a href="{{item2.url}}" class="list-group-item sub-item">
+                      {{ item2.title_in_menu }}</a>
+                   {% endfor %}
+                  </div>
+                {% else %}
+                  {% comment %} case 3a side menu item {% endcomment %}
+                  <a href="{{item.url}}" class="list-group-item{% if page.breadcrumb_paths contains item.menu_path %}{% unless item.menu_customization.force_inactive %} active{% endunless %}{% endif %}">{{ item.title_in_menu }}</a>
+                {% endif %}
+              {% endfor %}
+        
+    {% endif %}    
+  {% elsif page.menu %}
+    <!-- no pages have a menu without being in the top menu, won't come here -->
+  {% endif %}
+{% endunless %}
+        
             </div>
-            {% endif %}
         </div>
     </div>
 </div>

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_plugins/read.rb
----------------------------------------------------------------------
diff --git a/docs/_plugins/read.rb b/docs/_plugins/read.rb
index ae0ba75..23cf0f2 100644
--- a/docs/_plugins/read.rb
+++ b/docs/_plugins/read.rb
@@ -60,9 +60,9 @@ module JekyllRead
     def render(context)
       filename = @text.strip
       filename = context[filename] || filename
-
       # Pathname API ignores first arg below if second is absolute
-      file = Pathname.new(File.dirname(context['page']['path'])) + filename
+      page = context['page'] || context.registers[:page]
+      file = Pathname.new(File.dirname(page['path'])) + filename
       file = file.cleanpath
       # is there a better way to trim a leading / ?
       file = file.relative_path_from(Pathname.new("/")) unless file.relative?

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/_plugins/site_structure.rb
----------------------------------------------------------------------
diff --git a/docs/_plugins/site_structure.rb b/docs/_plugins/site_structure.rb
index e943730..70a07b6 100644
--- a/docs/_plugins/site_structure.rb
+++ b/docs/_plugins/site_structure.rb
@@ -1,22 +1,114 @@
+#
 # Builds a hierarchical structure for the site, based on the YAML front matter of each page
-# Starts from a page called "index.md", and follows "children" links in the YAML front matter
+#
+# Starts from a page called "/index.md" (or the value of `root_menu_page` in _config.yml),
+# and follows `children` links in the YAML front matter, building up a variable called `data` 
+# on `site` and on each referred `page`.
+#
+# In Ruby data is in page.data['menu'], but in templates `page.data` is promoted, 
+# so you should just refer to things in markdown as `{{ page.menu }}`.
+#
+# Each `page.menu` entry will contain:
+# * `url` - URL (relative or absolute) to link to
+# * `title_in_menu` - title to show
+# * `menu_path` - the path of this page for the purposes of looking in breadcrumbs (usually page.path, unless overriden) 
+# * `breadcrumbs_pages` - page items for ancestor items (and self)
+# * `breadcrumbs_paths` - paths of breadcrumb pages (useful for `if .. contains` jekyll tests)
+# * `menu_parent` - the parent menu which contains this page
+# * `menu_customization` - a hash of customization set in front matter or in children (can be any data you like)
+# * (in addition the entry may *be* the actual page object when the item is a page whose menu is not overridden)
+#
+# To build, set `children` as a list of either strings (the relative or absolute path to the child .md file),
+# or as maps with a `path` or `link` (absolute URL) key, a `title` (optional for `path`, to override the title from the file),
+# and (for `path` only) an optional `menu` block (to override the menu inherited from the `children` records in file),
+# `menu_customization` to set arbitrary data available (e.g. for templates to use when styling),
+# `href_path` to specify that an actual click should send to a different page than was used to produce the menu.
+#
+# For instance:
+#
+#children:
+#- child.md
+#- { path: child.md }  # identical to above
+#- { path: child.md, title: "Child with New Name" }  # overriding name
+#- { path: child.md, menu: [ { path: subchild.md, title: "Sub-Child with New Name" } ], href_path: subchild.md }  # custom sub-menu with custom title, and click on main sends to submenu 
+#- { path: child.md, menu: null }  # suppress sub-menu (note `null` not `nil` because this is yaml)
+#
+# The menu is automatically generated for all files referenced from the root menu.
+# You can also set `breadcrumbs` as a list of paths in a page to force breadcrumbs, and
+# `menu_proxy_for` to have `menu_path` set differently to the usual `path` (highlight another page in a menu via breadcrumbs)
+# or `menu_parent` to a path to the menu which should be the parent of the current node.
+# 
+# The hash `menu_customization` allows you to pass arbitrary data around, e.g. for use in styling.
+# 
+# Additionally URL rewriting is done if a path map is set in _config.yaml,
+# with `path: { xxx: /new_xxx }` causing `/xxx/foo.html` to be rewritten as `/new_xxx/foo.html`.
+#
 module SiteStructure
- 
-  BROOKLYN_WEBSITE_ROOT = "/website/index.md" unless defined? BROOKLYN_WEBSITE_ROOT
+
+  DEBUG = false
+
+  require 'yaml'  
+#  require 'pp'
+
+  class RewritePaths < Liquid::Tag
+    def initialize(tag_name, text, tokens)
+      super
+      @text = text
+    end
+    def render(context)
+      page = context['page']
+      site = context['site']
+      RewritePaths.rewrite_paths(site, page)
+    end
+    
+    def self.rewrite_paths(site, page)
+      path = page['path']
+      page_hash = (page.is_a? Hash) ? page : page.data
+      # set url_basedir and apply path mapping
+      page_hash['url_basedir'] = File.dirname(path)+"/"
+      page_hash['url_basedir'].prepend("/") unless page_hash['url_basedir'].start_with? "/"
+      
+      config_hash = (site.is_a? Hash) ? site : site.config
+      
+      if ((config_hash['path']) && (config_hash['path'].is_a? Hash))
+        config_hash['path'].each {|key, value| 
+          if (path.start_with?(key))
+            if ((!page.is_a? Hash) && page.url)
+              page.url.slice!("/"+key)
+              page.url.prepend(value)
+            end
+            
+            page_hash['url_basedir'].slice!("/"+key)
+            page_hash['url_basedir'].prepend(value)
+          end
+        }
+      end
+      
+      nil
+    end
+  end
+  
+  Liquid::Template.register_tag('rewrite_paths', SiteStructure::RewritePaths)
+
   
   class Generator < Jekyll::Generator
-    def find_page_with_path_absolute_or_relative_to(site, path, referrent, structure_processed_pages)
+
+    @@verbose = false;
+    
+    def self.find_page_with_path_absolute_or_relative_to(site, path, referrent, structure_processed_pages)
       uncleaned_path = path
       
       # Pathname API ignores first arg below if second is absolute
+#      puts "converting #{path} wrt #{referrent ? referrent.path : ""}"
       file = Pathname.new(File.dirname(referrent ? referrent.path : "")) + path
+      file += "index.md" if file.to_s.end_with? "/"
       file = file.cleanpath
       # is there a better way to trim a leading / ?
       file = file.relative_path_from(Pathname.new("/")) unless file.relative?
       path = "#{file}"
         
       # look in our cache        
-      page = structure_processed_pages.detect { |item| item['path'] == path }
+      page = structure_processed_pages[path]
       return page if page != nil
       
       # look in site cache
@@ -28,91 +120,183 @@ module SiteStructure
 
       unless page
         # could not load it from pages, look on disk
-                 
-        raise "No such file #{path} in site_structure call (from #{referrent ? referrent.path : ""})" unless file.exist?
-#        puts "INFO: reading excluded file #{file} for site structure generation"
-        page = Jekyll::Page.new(site, site.source, File.dirname(file), File.basename(file))
-        
-        throw "Could not find a page called: #{path} (referenced from #{referrent ? referrent.path : "root"})" unless page
-      end
 
-      # now apply standard clean-up
-      if (page.url.start_with?("/website"))
-        page.url.slice!("/website")
-        page.url.prepend(site.config['path']['website'])
-      end
-      if (page.url.start_with?("/guide"))
-        page.url.slice!("/guide")
-        page.url.prepend(site.config['path']['guide'])
+        if file.exist?                 
+          puts "INFO: reading excluded file #{file} for site structure generation" if SiteStructure::DEBUG
+          page = Jekyll::Page.new(site, site.source, File.dirname(file), File.basename(file))
+          # make sure right url is set
+          RewritePaths.rewrite_paths(site, page)
+        end
+ 
+        unless page
+          raise "No such file #{path} in site_structure call (from #{referrent ? referrent.path : ""})" unless SiteStructure::DEBUG
+          puts "Could not find a page called: #{path} (referenced from #{referrent ? referrent.path : "root"}); skipping"
+          return nil
+        end
       end
+      
       # and put in cache
-      structure_processed_pages << page
+      structure_processed_pages[path] = page
  
       page     
     end
 
     def generate(site)
-      structure_processed_pages = []
-      root_page = find_page_with_path_absolute_or_relative_to(site, SiteStructure::BROOKLYN_WEBSITE_ROOT, nil, structure_processed_pages)
-      navgroups = root_page.data['navgroups']
-      navgroups.each do |ng|
-        ng['page'] = find_page_with_path_absolute_or_relative_to(site, ng['page'], root_page, structure_processed_pages)
-        if not ng['title_in_menu']
-          ng['title_in_menu'] = ng['title'].capitalize
-        end
-      end
-      site.data['navgroups'] = navgroups
-      site.data['structure'] = gen_structure(site, SiteStructure::BROOKLYN_WEBSITE_ROOT, nil, navgroups, structure_processed_pages)
+      # rewrite paths
+      site.pages.each { |p| RewritePaths.rewrite_paths(site, p) }
+      structure_processed_pages = {}
+      # process root page
+      root_menu_page = site.config['root_menu_page']
+      puts "site_structure processing root menu page #{root_menu_page}" if @@verbose
+      site.data.merge!( Generator.gen_structure(site, { 'path' => root_menu_page }, nil, [], [], structure_processed_pages).data ) if root_menu_page
+      # process all pages
+      puts "site_structure now processing all pages" if @@verbose
+      site.pages.each { |p| 
+        Generator.gen_structure(site, { 'path' => p.path }, nil, [], [], structure_processed_pages) if (p.path.end_with? ".md") && (!p['menu_processed'])
+      }
+      site.data['structure_processed_pages'] = structure_processed_pages
+#      puts "ROOT menu is #{site.data['menu']}"
+#      puts "PAGE menu is #{structure_processed_pages['website/documentation/index.md'].data['menu']}"
+# (but note, in the context hash map 'data' on pages is promoted, so you access it like {{ page.menu }})
     end
 
-    def render_liquid(site, page, content)
+    # processes liquid tags, e.g. in a link or path object
+    def self.render_liquid(site, page, content)
+      return content unless page
       info = { :filters => [Jekyll::Filters], :registers => { :site => site, :page => page } }
       page.render_liquid(content, site.site_payload, info)
     end
-        
-    def gen_structure(site, pagename, parent, navgroups, structure_processed_pages)
-      page = find_page_with_path_absolute_or_relative_to(site, pagename, parent, structure_processed_pages)
-      
-      # My navgroup is (first rule matches):
-      # 1. what I have explicitly declared
-      # 2. if I find my path referred to in the global navgroup list
-      # 3. my parent's navgroup
-      unless page.data['navgroup']
-        match = navgroups.detect { |ng| ng['page'] == page }
-        if match
-          page.data['navgroup'] = match['id']
-        elsif parent
-          page.data['navgroup'] = parent.data['navgroup']
+    
+    def self.gen_structure(site, item, parent, breadcrumb_pages_in, breadcrumb_paths_in, structure_processed_pages)
+      puts "gen_structure #{item} from #{parent ? parent.path : 'root'} (#{breadcrumb_paths_in})" if @@verbose
+      breadcrumb_pages = breadcrumb_pages_in.dup
+      breadcrumb_paths = breadcrumb_paths_in.dup
+      if (item.is_a? String)
+        item = { 'path' => item }
+      end
+      if (item['path'])      
+        page = find_page_with_path_absolute_or_relative_to(site, render_liquid(site, parent, item['path']), parent, structure_processed_pages)
+        # if nil and find_page doesn't raise, we are in debug mode, silently ignore
+        return nil unless page
+        # build up the menu info
+        if (item.length==1 && !page['menu_processed'])
+          puts "setting up #{item} from #{page.path} as original" if @@verbose
+          data = page.data
+          result = page
+        else
+          puts "setting up #{item} from #{page.path} as copy" if @@verbose
+          # if other fields are set on 'item' then we are overriding, so we have to take a duplicate
+          unless page['menu_processed']
+            # force processing if not yet processed, breadcrumbs etc set from that page
+            puts "making copy of #{page.path}" if @@verbose
+            page = gen_structure(site, "/"+page.path, parent, breadcrumb_pages_in, breadcrumb_paths_in, structure_processed_pages)
+            puts "copy is #{page.path}" if @@verbose
+          end
+          data = page.data.dup
+          data['data'] = data
+          result = data
+        end 
+        data['path'] = page.path
+        if item['href_path']
+          href_page = find_page_with_path_absolute_or_relative_to(site, render_liquid(site, page, item['href_path']), parent, structure_processed_pages)
+        else
+          href_page = page
         end
+        data['url'] = href_page.url
+        puts "data is #{data}" if @@verbose
+        data['page'] = page
+        breadcrumb_pages << page
+        breadcrumb_paths << page.path
+      elsif (item['link'])
+        puts "setting up #{item} as link" if @@verbose
+        link = render_liquid(site, parent, item['link'])
+        data = { 'link' => link, 'url' => link }
+        breadcrumb_pages << data
+        breadcrumb_paths << data['link']
+        data['data'] = data
+        result = data
+      else
+        raise "Link to #{item} in #{parent ? parent.path : nil} must have link or path"
       end
-            
-      # Figure out second level menu
-      # If there's no parent => I'm at the top level, so no action
-      # If there's a parent, but parent has no parent => I'm at second level, so set second-level menu
-      # Otherwise, use the parent's second level menu
-      if parent && !parent.data['parent']
-        page.data['menu2parent'] = page
-        page.data['menu2'] = page.data['children']
-      elsif parent && parent.data['parent']
-        page.data['menu2parent'] = parent.data['menu2parent']
-        page.data['menu2'] = parent.data['menu2']
+
+      data['menu_customization'] = {}.merge(data['menu_customization'] || {}).merge(item['menu_customization'] || {})
+      
+      data['breadcrumb_pages'] ||= breadcrumb_pages
+      data['breadcrumb_paths'] ||= breadcrumb_paths
+      data['menu_parent'] ||= parent
+      
+      data['title_in_menu'] = render_liquid(site, parent, item['title_in_menu'] || item['title'] || data['title_in_menu'] || data['title'])
+#      puts "built #{data}, now looking at children"
+
+      # if already processed then return now that we have set custom item overrides (don't recurse through children)
+      return result if data['menu']
+      
+      data['menu_path'] = page.path if page
+      
+      if data['menu_proxy_for']
+        menu_proxy_for = gen_structure(site, { 'path' => data['menu_proxy_for'], 'no_copy' => "because breadcrumbs won't be right" }, page, [], [], structure_processed_pages)
+        raise "missing menu_proxy_for #{data['menu_proxy_for']} in #{page.path}" unless menu_proxy_for
+        data['menu_path'] = menu_proxy_for['path']
+        # copy other data across
+        data.merge!(menu_proxy_for.select {|key, value| ['breadcrumb_paths', 'breadcrumb_pages', 'menu', 'title_in_menu', 'menu_parent', 'menu_customization'].include?(key) })
       end
       
-      page.data['parent'] = parent
-      if page.data['children']
-        page.data['children'].each do |c|
-          if c['path']
-            # links to another Jekyll site-structured page
-            c['reference'] = gen_structure(site, render_liquid(site, page, c['path']), page, navgroups, structure_processed_pages)
-          elsif c['link']
-            # links to a non-site-structured page, on this site or elsewhere
-            # allow title and link to use vars and tags (liquid processing)
-            c['reference'] = { 'url' => render_liquid(site, page, c['link']), 'title' => render_liquid(site, page, c['title']) }
+      if data['breadcrumbs']
+        # if custom breadcrumbs set on page, use them instead
+        breadcrumb_pages = data['breadcrumb_pages'] = data['breadcrumbs'].collect { |path|
+          result = find_page_with_path_absolute_or_relative_to(site, render_liquid(site, parent, path), page, structure_processed_pages)
+          raise "missing breadcrumb #{path} in #{page.path}" unless result
+          result
+        }
+        breadcrumb_paths = data['breadcrumb_paths'] = data['breadcrumb_pages'].collect { |p| p.path }
+      end
+
+      if data['menu_parent'] 
+        if data['menu_parent'].is_a? String
+          # if custom menu_parent was set as a string then load it
+          parent_result = find_page_with_path_absolute_or_relative_to(site, render_liquid(site, parent, data['menu_parent']), page, structure_processed_pages)        
+          raise "missing parent #{data['menu_parent']} in #{page['path']}" unless parent_result
+          data['menu_parent'] = parent_result
+          if !data['breadcrumbs']
+            # TODO should we inherit actual menu parent breadcrumbs if not set on page?
+          end
+        end
+      else
+        # set menu_parent from breadcrumbs if not set (e.g. we are loading an isolated page)
+        data['menu_parent'] = page['breadcrumb_pages'][-1]
+      end
+
+      if (data['children'])
+        data['menu'] = []
+        puts "children of #{data['path']} - #{data['children']}" if @@verbose
+        data['children'].each do |child|
+          sub = gen_structure(site, child, page, breadcrumb_pages, breadcrumb_paths, structure_processed_pages)
+          if sub
+            if (!(child.is_a? String) && child.has_key?('menu'))
+              # process custom menu override
+              sub['menu'] = child['menu']
+              if (sub['menu'] != nil)
+                if sub['menu'].is_a? String
+                  sub['menu'] = YAML.load(render_liquid(site, page, sub['menu'])) if sub['menu'].is_a? String
+                end
+                sub['menu'] = sub['menu'].collect { |mi| 
+                  gen_structure(site, mi, page, breadcrumb_pages, breadcrumb_paths, structure_processed_pages)
+                }
+                sub['menu'].compact!
+              end
+            end
+            data['menu'] << sub
+            puts "sub is #{sub['url']}" if @@verbose
+          else
+            raise "could not find #{child} in #{page.path}"
           end
         end
+        puts "end children of #{data['path']}" if @@verbose
       end
       
-      page
+      data['menu_processed']=true
+      puts "done #{item}" if @@verbose
+      result
     end
   end
 end

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/guide/concepts/advanced-concepts.md
----------------------------------------------------------------------
diff --git a/docs/guide/concepts/advanced-concepts.md b/docs/guide/concepts/advanced-concepts.md
new file mode 100644
index 0000000..1b36170
--- /dev/null
+++ b/docs/guide/concepts/advanced-concepts.md
@@ -0,0 +1,14 @@
+---
+title: Advanced Concepts
+layout: guide-normal
+toc: ../guide_toc.json
+categories: [use, guide, defining-applications]
+---
+
+
+
+
+
+
+
+

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/guide/concepts/application-parent-membership.md
----------------------------------------------------------------------
diff --git a/docs/guide/concepts/application-parent-membership.md b/docs/guide/concepts/application-parent-membership.md
new file mode 100644
index 0000000..18f1a54
--- /dev/null
+++ b/docs/guide/concepts/application-parent-membership.md
@@ -0,0 +1,25 @@
+---
+title: Application, Parent and Membership
+layout: guide-normal
+toc: ../guide_toc.json
+categories: [use, guide, defining-applications]
+---
+
+All entities have a ***parent*** entity, which creates and manages it, with one important exception: *applications*.
+Application entities are the top-level entities created and managed externally, manually or programmatically.
+
+Applications are typically defined in Brooklyn as an ***application descriptor***. 
+This is a Java class specifying the entities which make up the application,
+by extending the class ``AbstractApplication``, and specifying how these entities should be configured and managed.
+
+All entities, including applications, can be the parent of other entities. 
+This means that the "child" is typically started, configured, and managed by the parent.
+For example, an application may be the parent of a web cluster; that cluster in turn is the parent of web server processes.
+In the management console, this is represented hierarchically in a tree view.
+
+A parallel concept is that of ***membership***: in addition to one fixed parent,
+and entity may be a ***member*** of any number of special entities called ***groups***.
+Membership of a group can be used for whatever purpose is required; 
+for example, it can be used to manage a collection of entities together for one purpose 
+(e.g. wide-area load-balancing between locations) even though they may have been
+created by different parents (e.g. a multi-tier stack within a location).

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/guide/concepts/configuration-sensor-effectors.md
----------------------------------------------------------------------
diff --git a/docs/guide/concepts/configuration-sensor-effectors.md b/docs/guide/concepts/configuration-sensor-effectors.md
new file mode 100644
index 0000000..5d57b4d
--- /dev/null
+++ b/docs/guide/concepts/configuration-sensor-effectors.md
@@ -0,0 +1,43 @@
+---
+title: Configuration, Sensors and Effectors
+layout: guide-normal
+toc: ../guide_toc.json
+categories: [use, guide, defining-applications]
+---
+
+### Configuration
+
+All entities contain a map of config information. This can contain arbitrary values, typically keyed under static ``ConfigKey`` fields on the ``Entity`` sub-class. These values are inherited, so setting a configuration value at the
+application level will make it available in all entities underneath unless it is overridden.
+
+Configuration is propagated when an application "goes live" (i.e. it becomes "managed", either explicitly or when its ``start()`` method is invoked), so config values must be set before this occurs. 
+
+Configuration values can be specified in a configuration file (``~/.brooklyn/brooklyn.properties``)
+to apply universally, and/or programmatically to a specific entity and its descendants 
+by calling `.configure(KEY, VALUE)` in the entity spec when creating it.
+There is also an ``entity.setConfig(KEY, VALUE)`` method.
+
+Additionally, many common configuration parameters are available as "flags" which can be supplied as Strings when constructing
+then entity, in the form
+``EntitySpec.createË™(MyEntity.class).configure("config1", "value1").configure("config2", "value2")``. 
+
+Documentation of the flags available for individual entities can normally be found in the javadocs. 
+The ``@SetFromFlag`` annotations on ``ConfigKey`` static field definitions
+in the entity's interface is the recommended mechanism for exposing configuration options.
+
+
+### Sensors and Effectors
+
+***Sensors*** (activity information and notifications) and ***effectors*** (operations that can be invoked on the entity) are defined by entities as static fields on the ``Entity`` subclass.
+
+Sensors can be updated by the entity or associated tasks, and sensors from an entity can be subscribed to by its parent or other entities to track changes in an entity's activity.
+
+Effectors can be invoked by an entity's parent remotely, and the invoker is able to track the execution of that effector. Effectors can be invoked by other entities, but use this functionality with care to prevent too many managers!
+
+An entity consists of a Java interface (used when interacting with the entity) and a Java class. For resilience. it is recommended to store 
+the entity's state in attributes (see `getAttribute(AttributeKey)``). If internal fields can be used then the data will be lost on brooklyn 
+restart, and may cause problems if the entity is to be moved to a different brooklyn management node.
+
+Next: [Advanced Concepts]({{site.path.guide}}/use/guide/defining-applications/advanced-concepts.html).
+See also: [Management > Sensors and Effectors]({{site.path.guide}}/use/guide/management/index.html#sensors-and-effectors).
+

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/guide/concepts/dependent-configuration.md
----------------------------------------------------------------------
diff --git a/docs/guide/concepts/dependent-configuration.md b/docs/guide/concepts/dependent-configuration.md
new file mode 100644
index 0000000..0ed60ff
--- /dev/null
+++ b/docs/guide/concepts/dependent-configuration.md
@@ -0,0 +1,34 @@
+---
+title: Dependent Configuration
+layout: guide-normal
+toc: ../guide_toc.json
+categories: [use, guide, defining-applications]
+---
+
+Under the covers Brooklyn has a sophisticated sensor event and subscription model, but conveniences around this model make it very simple to express cross-entity dependencies. Consider the example where Tomcat instances need to know the URL of a database (or a set of URLs to connect to a Monterey processing fabric, or other entities)
+
+{% highlight java %}
+setConfiguration(UsesJava.JAVA_OPTIONS, ImmutableMap.of("mysql.url", 
+	    attributeWhenReady(mysql, MySqlNode.MY_SQL_URL) ))
+{% endhighlight %}
+
+The ``attributeWhenReady(Entity, Sensor)`` call (a static method on the class ``DependentConfiguration``)
+causes the configuration value to be set when that given entity's attribue is ready. 
+In the example, ``attributeWhenReady()`` causes the JVM system property ``mysql.url`` to be set to the value of the ``MySqlNode.MY_SQL_URL`` sensor from ``mysql`` when that value is ready. As soon as the database URL is announced by the MySql entity, the configuration value will be available to the Tomcat cluster. 
+
+By default "ready" means being *set* (non-null) and, if appropriate, *non-empty* (for collections and strings) or *non-zero* (for numbers). Formally the interpretation of ready is that of "Groovy truth" defined by an ``asBoolean()`` method on the class and in the Groovy language extensions. 
+
+You can customize "readiness" by supplying a ``Predicate`` (Google common) or ``Closure`` (Groovy) in a third parameter. 
+This evaluates candidate values reported by the sensor until one is found to be ``true``. 
+For example, passing ``{ it.size()>=3 }`` as the readiness argument would require at least three management plane URLs.
+
+More information on this can be found in the javadoc for ``DependentConfiguration``,
+along with a few other methods such as ``valueWhenAttributeReady`` which allow post-processing of an attribute value.
+
+Note that if the value of ``CONFIG_KEY`` passed to ``Entity.getConfig`` is a Closure or Task (such as returned by ``attributeWhenReady``),
+the first access of ``Entity.getConfig(CONFIG_KEY)`` will block until the task completes.
+Typically this does the right thing, blocking when necessary to generate the right start-up sequence
+without the developer having to think through the order, but it can take some getting used to.
+Be careful not to request config information until really necessary (or to use non-blocking "raw" mechanisms),
+and in complicated situations be ready to attend to circular dependencies.
+The management console gives useful information for understanding what is happening and resolving the cycle.

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/guide/concepts/entities.md
----------------------------------------------------------------------
diff --git a/docs/guide/concepts/entities.md b/docs/guide/concepts/entities.md
new file mode 100644
index 0000000..26981e7
--- /dev/null
+++ b/docs/guide/concepts/entities.md
@@ -0,0 +1,23 @@
+---
+title: Entities
+layout: guide-normal
+toc: ../guide_toc.json
+categories: [use, guide, defining-applications]
+---
+
+The central concept in a Brooklyn deployment is that of an ***entity***. 
+An entity represents a resource under management, either *base* entities (individual machines or software processes) 
+or logical collections of these entities.
+
+Fundamental to the processing model is the capability of entities to be the *parent* of other entities (the mechanism by which collections are formed), 
+with every entity having a single parent entity, up to the privileged top-level ***application*** entity.
+
+Entities are code, so they can be extended, overridden, and modified. Entities can have events, operations, and processing logic associated with them, and it is through this mechanism that the active management is delivered.
+
+The main responsibilities of an entity are:
+
+- Provisioning the entity in the given location or locations
+- Holding configuration and state (attributes) for the entity
+- Reporting monitoring data (sensors) about the status of the entity
+- Exposing operations (effectors) that can be performed on the entity
+- Hosting management policies and tasks related to the entity

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/guide/concepts/execution.md
----------------------------------------------------------------------
diff --git a/docs/guide/concepts/execution.md b/docs/guide/concepts/execution.md
new file mode 100644
index 0000000..4bb4875
--- /dev/null
+++ b/docs/guide/concepts/execution.md
@@ -0,0 +1,36 @@
+---
+title: Execution
+layout: guide-normal
+toc: ../guide_toc.json
+categories: [use, guide, defining-applications]
+---
+
+All processing, whether an effector invocation or a policy cycle, are tracked as ***tasks***. This allows several important capabilities:
+
+*	active and historic processing can be observed by operators
+*	the invocation context is available in the thread, to check entitlement (permissions) and maintain a
+hierarchical causal chain even when operations are run in parallel
+*	processing can be managed across multiple management nodes
+
+Some executions create new entities, which can then have tasks associated with them, and the system will record, for example, that a start efector on the new entity is a task associated with that entity, with that task
+created by a task associated with a different entity.
+
+The execution of a typical overall start-up sequence is shown below:
+
+[![Brooklyn Flow Diagram](brooklyn-flow-websequencediagrams.com-w400.png "Brooklyn Flow Diagram" )](brooklyn-flow-websequencediagrams.com.png)
+
+
+## Integration
+
+One vital aspect of Brooklyn is its ability to communicate with the systems it starts. This is abstracted using a ***driver*** facility in Brooklyn, where a
+driver describes how a process or service can be installed and managed using a particular technology.
+
+For example, a ``TomcatServer`` may implement start and other effectors using a ``TomcatSshDriver`` which inherits from ``JavaSoftwareProcessSshDriver`` (for JVM and JMX start confguration), inheriting from ``AbstractSoftwareProcessSshDriver``
+(for SSH scripting support).
+
+Particularly for sensors, some technologies are used so frequently that they are
+packaged as ***feeds*** which can discover their configuration (including from drivers). These include JMX and HTTP (see ``JmxFeed`` and ``HttpFeed``).
+
+Brooklyn comes with entity implementations for a growing number of commonly used systems, including various web application servers, databases and NoSQL data stores, and messaging systems. See: [Extras]({{site.path.guide}}/use/guide/extras/index.html).
+
+

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/guide/concepts/index.md
----------------------------------------------------------------------
diff --git a/docs/guide/concepts/index.md b/docs/guide/concepts/index.md
new file mode 100644
index 0000000..5c81c2d
--- /dev/null
+++ b/docs/guide/concepts/index.md
@@ -0,0 +1,21 @@
+---
+title: Brooklyn Concepts
+title_in_menu: Brooklyn Concepts
+layout: website-normal
+children:
+- entities.md
+- application-parent-membership.md
+- configuration-sensor-effectors.md
+- lifecycle-managementcontext.md
+- dependent-configuration.md
+- location.md
+- policies.md
+- execution.md
+---
+
+This introduces brooklyn and describes how it simplifies the deployment and management of big applications. It is
+intended for people who are using brooklyn-supported application components (such as web/app servers, data stores)
+to be able to use brooklyn to easily start their application in multiple locations with off-the-shelf management
+policies.
+
+{% include list-children.html %}

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/guide/concepts/lifecycle-managementcontext.md
----------------------------------------------------------------------
diff --git a/docs/guide/concepts/lifecycle-managementcontext.md b/docs/guide/concepts/lifecycle-managementcontext.md
new file mode 100644
index 0000000..3495e17
--- /dev/null
+++ b/docs/guide/concepts/lifecycle-managementcontext.md
@@ -0,0 +1,44 @@
+---
+title: Lifecycle and ManagementContext
+layout: guide-normal
+toc: ../guide_toc.json
+categories: [use, guide, defining-applications]
+---
+
+Under-the-covers, at heart of the brooklyn management plane is the ``ManagementContext``. 
+This is started automatically when using launching an application using the brooklyn CLI. For programmatic use, see 
+``BrooklynLauncher.newLauncher().launch()``.
+
+A Brooklyn deployment consists of many entities in a hierarchical tree, with the privileged *application* entity at the top level.
+
+An application entity (``Application`` class) is responsible for starting the deployment of all its child entities (i.e. the entire entity tree under its ownership).
+
+An ``Application``'s ``start()`` method begins provisioning the child entities of the application (and their entities, recursively). 
+
+Provisioning of entities typically happens in parallel automatically,
+although this can be customized. This is implemented as ***tasks*** which are tracked by the management plane and is visible in the [web-based management console]({{site.path.guide}}/use/guide/management/index.html#console).
+
+Customized provisioning can be useful where two starting entities depend on each other. For example, it is often necessary to delay start of one entity until another entity reaches a certain state, and to supply run-time information about the latter to the former.
+
+<!-- TODO ambiguous language; need a better description of the "manage" lifecycle -->
+When new entities are created, the entity is wired up to an application by giving it a parent. The entity is then explicitly "managed", which allows other entities to discover it.
+
+Typically a Brooklyn deployment has a single management context which records:
+
+*   all entities under management that are reachable by the application(s) via the parent-child relationships,
+*	the state associated with each entity,
+*	subscribers (listeners) to sensor events arising from the entities,
+*	active tasks (jobs) associated with any the entity,
+*	which Brooklyn management node is mastering (managing) each entity.
+
+<!-- TODO Distributed brooklyn not yet supported; needs clarification in docs -->
+
+In a multi-location deployment, management operates in all regions, with brooklyn entity instances being mastered in the relevant region.
+
+When management is distributed a Brooklyn deployment may consist of multiple Brooklyn management nodes each with a ``ManagementContext`` instance.
+
+<!-- TODO - Clarify the following statements.
+The management context entity forms part of the management plane. 
+The management plane is responsible for the distribution of the ``Entity`` instances across multiple machines and multiple locations, 
+tracking the transfer of events (subscriptions) between ``Entity`` instances, and the execution of tasks (often initiated by management policies).
+-->

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/guide/concepts/location.md
----------------------------------------------------------------------
diff --git a/docs/guide/concepts/location.md b/docs/guide/concepts/location.md
new file mode 100644
index 0000000..2a06cac
--- /dev/null
+++ b/docs/guide/concepts/location.md
@@ -0,0 +1,22 @@
+---
+title: Location
+layout: guide-normal
+toc: ../guide_toc.json
+categories: [use, guide, defining-applications]
+---
+
+<!-- TODO, Clarify is how geographical location works.
+-->
+
+Entities can be provisioned/started in the location of your choice. Brooklyn transparently uses [jclouds](http://www.jclouds.org) to support different cloud providers and to support BYON (Bring Your Own Nodes). 
+
+The implementation of an entity (e.g. Tomcat) is agnostic about where it will be installed/started. When writing the application definition specify the location or list of possible locations (``Location`` instances) for hosting the entity.
+
+``Location`` instances represent where they run and indicate how that location (resource or service) can be accessed.
+
+For example, a ``JBoss7Server`` will usually be running in an ``SshMachineLocation``, which contains the credentials and address for sshing to the machine. A cluster of such servers may be running in a ``MachineProvisioningLocation``, capable of creating new ``SshMachineLocation`` instances as required.
+
+<!-- TODO, incorporate the following.
+
+The idea is that you could specify the location as AWS and also supply an image id. You could configure the Tomcat entity accordingly: specify the path if the image already has Tomcat installed, or specify that Tomcat must be downloaded/installed. Entities typically use _drivers_ (such as SSH-based) to install, start, and interact with their corresponding real-world instance. 
+-->

http://git-wip-us.apache.org/repos/asf/incubator-brooklyn/blob/a7a317bf/docs/guide/concepts/policies.md
----------------------------------------------------------------------
diff --git a/docs/guide/concepts/policies.md b/docs/guide/concepts/policies.md
new file mode 100644
index 0000000..ad8c0bb
--- /dev/null
+++ b/docs/guide/concepts/policies.md
@@ -0,0 +1,11 @@
+---
+title: Policies
+layout: guide-normal
+toc: ../guide_toc.json
+categories: [use, guide, defining-applications]
+---
+
+Policies perform the active management enabled by Brooklyn. Entities can have zero or more ``Policy`` instances attached to them. 
+
+Policies can subscribe to sensors from entities or run periodically, and
+when they run they can perform calculations, look up other values, and if deemed necessary invoke effectors or emit sensor values from the entity with which they are associated.


Mime
View raw message