aurora-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dles...@apache.org
Subject svn commit: r1623604 [1/2] - in /incubator/aurora/site: ./ publish/documentation/latest/client-commands/ publish/documentation/latest/clientv2/ publish/documentation/latest/committers/ publish/documentation/latest/configuration-reference/ publish/docum...
Date Tue, 09 Sep 2014 00:47:20 GMT
Author: dlester
Date: Tue Sep  9 00:47:19 2014
New Revision: 1623604

URL: http://svn.apache.org/r1623604
Log:
Updates README file about updating the website, as well as docs that appear on the website.

Added:
    incubator/aurora/site/publish/documentation/latest/clientv2/
    incubator/aurora/site/publish/documentation/latest/clientv2/index.html
    incubator/aurora/site/publish/documentation/latest/committers/
    incubator/aurora/site/publish/documentation/latest/committers/index.html
    incubator/aurora/site/publish/documentation/latest/developing-aurora-client/
    incubator/aurora/site/publish/documentation/latest/developing-aurora-client/index.html
    incubator/aurora/site/publish/documentation/latest/sla/
    incubator/aurora/site/publish/documentation/latest/sla/index.html
    incubator/aurora/site/source/documentation/latest/clientv2.md
    incubator/aurora/site/source/documentation/latest/committers.md
    incubator/aurora/site/source/documentation/latest/developing-aurora-client.md
    incubator/aurora/site/source/documentation/latest/sla.md
    incubator/aurora/site/source/layouts/documentation.erb
Modified:
    incubator/aurora/site/README.md
    incubator/aurora/site/publish/documentation/latest/client-commands/index.html
    incubator/aurora/site/publish/documentation/latest/configuration-reference/index.html
    incubator/aurora/site/publish/documentation/latest/configuration-tutorial/index.html
    incubator/aurora/site/publish/documentation/latest/contributing/index.html
    incubator/aurora/site/publish/documentation/latest/deploying-aurora-scheduler/index.html
    incubator/aurora/site/publish/documentation/latest/developing-aurora-scheduler/index.html
    incubator/aurora/site/publish/documentation/latest/user-guide/index.html
    incubator/aurora/site/publish/documentation/latest/vagrant/index.html
    incubator/aurora/site/source/documentation/latest/client-commands.md
    incubator/aurora/site/source/documentation/latest/configuration-reference.md
    incubator/aurora/site/source/documentation/latest/configuration-tutorial.md
    incubator/aurora/site/source/documentation/latest/contributing.md
    incubator/aurora/site/source/documentation/latest/deploying-aurora-scheduler.md
    incubator/aurora/site/source/documentation/latest/developing-aurora-scheduler.md
    incubator/aurora/site/source/documentation/latest/user-guide.md
    incubator/aurora/site/source/documentation/latest/vagrant.md

Modified: incubator/aurora/site/README.md
URL: http://svn.apache.org/viewvc/incubator/aurora/site/README.md?rev=1623604&r1=1623603&r2=1623604&view=diff
==============================================================================
--- incubator/aurora/site/README.md (original)
+++ incubator/aurora/site/README.md Tue Sep  9 00:47:19 2014
@@ -1,48 +1,46 @@
 # Apache Aurora Website
-This is the website for Apache Aurora (incubating). 
+This will generate the Apache Aurora available at http://aurora.apache.org. The content
+within the publish folder will be the actual deployed site.
 
 
-### Development Setup
-We use middleman to generate the website content from markdown and other 
-dynamic templates. The following steps assume you have a working 
-ruby environment setup
+## Setup
 
-	gem install bundler
-	bundle install
+		gem install bundler
+		bundle install
+		
 
-On Ubuntu 14.04 (Trusty Tahr) you'll instead want to use the system ruby 2.0:
+## Generating the site
+To generate the site one only needs to run `rake` after performing the setup
+tasks mentioned above. This will download the latest Apache Aurora documentation
+contained in the `docs` folder, integrate them into the site, and generate all
+other files within the source folder.
 
-    sudo apt-get install ruby2.0 ruby2.0-dev
-    gem2.0 install bundler --user
-    # Add this to your PATH persistently somehow
-    PATH="$HOME/.gem/ruby/2.0.0/bin:$PATH"
-    bundle install
+		rake
 
 
-### Generating the website
----
-To generate the static wesbite for Apache Aurora run the following commands
+## Development 
+To live edit the site run `rake dev` and then open a browser window to 
+http://localhost:4567/ . Any change you make to the sources dir will 
+be shown on the local dev site immediately. Errors will be shown in the 
+console you launched `rake dev` within.
 
-		bundle exec middleman build
 
+## Other available tasks
 
-### Live Development 
----
-Live development of the site enables automatic reload when changes are saved. 
-To enable run the following command and then open a browser and navigate to 
-[http://localhost:4567](http://localhost:4567/) 
+		rake build        # Build the website from source
+		rake clean        # Remove any temporary products
+		rake clobber      # Remove any generated file
+		rake dev          # Run the site in development mode
+		rake update_docs  # Update the latest docs from the Apache Aurora codebase
 
-		bundle exec middleman 
 
-
-### Publishing the Site
+## Publishing the Site
 The website uses svnpubsub. The publish folder contains the websites content
 and when committed to the svn repository it will be automatically deployed to 
 the live site. 
 
 
 ### Apache License
----
 Except as otherwise noted this software is licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0.html)
 
 Licensed under the Apache License, Version 2.0 (the "License");
@@ -55,4 +53,4 @@ Unless required by applicable law or agr
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
-limitations under the License.
+limitations under the License.
\ No newline at end of file

Modified: incubator/aurora/site/publish/documentation/latest/client-commands/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/client-commands/index.html?rev=1623604&r1=1623603&r2=1623604&view=diff
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/client-commands/index.html (original)
+++ incubator/aurora/site/publish/documentation/latest/client-commands/index.html Tue Sep  9 00:47:19 2014
@@ -387,7 +387,7 @@ manually. Defaults to <code>False</code>
 <li><code>--restart_threshold</code>: Defaults to <code>60</code>, the maximum number of
 seconds before a shard must move into the <code>RUNNING</code> state before
 it&rsquo;s considered a failure.</li>
-<li><code>--watch_secs</code>: Defaults to <code>30</code>, the minimum number of seconds a
+<li><code>--watch_secs</code>: Defaults to <code>45</code>, the minimum number of seconds a
 shard must remain in <code>RUNNING</code> state before considered a success.</li>
 </ul>
 

Added: incubator/aurora/site/publish/documentation/latest/clientv2/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/clientv2/index.html?rev=1623604&view=auto
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/clientv2/index.html (added)
+++ incubator/aurora/site/publish/documentation/latest/clientv2/index.html Tue Sep  9 00:47:19 2014
@@ -0,0 +1,527 @@
+<html>
+    <head>
+        <meta charset="utf-8">
+        <title>Apache Aurora</title>
+		    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+		    <meta name="description" content="">
+		    <meta name="author" content="">
+
+		    <link href="/assets/css/bootstrap.css" rel="stylesheet">
+		    <link href="/assets/css/bootstrap-responsive.min.css" rel="stylesheet">
+		    <link href="/assets/css/main.css" rel="stylesheet">
+				
+		    <!-- JS -->
+		    <script type="text/javascript" src="/assets/js/jquery-1.10.1.min.js"></script>
+		    <script type="text/javascript" src="/assets/js/bootstrap-dropdown.js"></script>
+		
+				<!-- Analytics -->
+				<script type="text/javascript">
+					  var _gaq = _gaq || [];
+					  _gaq.push(['_setAccount', 'UA-45879646-1']);
+					  _gaq.push(['_setDomainName', 'apache.org']);
+					  _gaq.push(['_trackPageview']);
+
+					  (function() {
+					    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+					    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+					    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+					  })();
+				</script>
+	</head>
+    <body>	
+      <div class="navbar navbar-static-top">
+  <div class="navbar-inner">
+    <div class="container">
+	    <a href="/" class="logo"><img src="/assets/img/aurora_logo.png" alt="Apache Aurora logo" /></a>
+      <ul class="nav">
+				<li><a href="/documentation/latest/">Documentation</a></li>
+        <li><a href="/downloads/">Download</a></li>
+        <li><a href="/community">Community</a></li>
+      </ul>
+    </div>
+  </div>
+</div>
+
+<div class="container">
+<!-- magical breadcrumbs -->
+<ul class="breadcrumb">
+  <li>
+    <div class="dropdown">
+      <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a>
+      <ul class="dropdown-menu" role="menu">
+        <li><a href="http://www.apache.org">Apache Homepage</a></li>
+        <li><a href="http://www.apache.org/licenses/">Apache License</a></li>
+        <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>  
+        <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+        <li><a href="http://www.apache.org/security/">Security</a></li>
+      </ul>
+    </div>
+  </li>
+  <li><span class="divider">&bull;</span></li>
+  <li><a href="http://incubator.apache.org">Apache Incubator</a></li>
+  <li><span class="divider">&bull;</span></li>
+  <li><a href="http://aurora.incubator.apache.org">Apache Aurora</a></li>
+</ul>
+<!-- /breadcrumb -->
+	
+      <div class="container">
+        <h1 id="aurora-client-v2">Aurora Client v2</h1>
+
+<h2 id="overview">Overview</h2>
+
+<p>Our goal is to replace the current Aurora command-line client. The
+current client suffers from an early monolithic structure, and a long
+development history of rapid unplanned evolution.</p>
+
+<p>In addition to its internal problems, the current Aurora client is
+confusing for users. There are several different kinds of objects
+manipulated by the Aurora command line, and the difference between
+them is often not clear. (What&rsquo;s the difference between a job and a
+configuration?) For each type of object, there are different commands,
+and it&rsquo;s hard to remember which command should be used for which kind
+of object.</p>
+
+<p>Instead of continuing to let the Aurora client develop and evolve
+randomly, it&rsquo;s time to take a principled look at the Aurora command
+line, and figure out how to make our command line processing make
+sense. At the same time, the code needs to be cleaned up, and divided
+into small comprehensible units based on a plugin architecture.</p>
+
+<p>Doing this now will give us a more intuitive, consistent, and easy to
+use client, as well as a sound platform for future development.</p>
+
+<h2 id="goals">Goals</h2>
+
+<ul>
+<li>A command line tool for interacting with Aurora that is easy for
+users to understand.</li>
+<li>A noun/verb command model.</li>
+<li>A modular source-code architecture.</li>
+<li>Non-disruptive transition for users.</li>
+</ul>
+
+<h2 id="non-goals">Non-Goals</h2>
+
+<ul>
+<li>The most important non-goal is that we&rsquo;re not trying to redesign the
+Aurora scheduler, the Aurora executor, or any of the peripheral tools
+that the Aurora command line interacts with; we only want to create a
+better command line client.</li>
+<li>We do not want to change thermos, mesos, hadoop, etc.</li>
+<li>We do not want to create new objects that users will work with to
+interact with Mesos or Aurora.</li>
+<li>We do not want to change Aurora job configuration files or file formats.</li>
+<li>We do not want to change the Aurora API.</li>
+<li>We don&rsquo;t want to boil the ocean: there are many things that we could
+include in the scope of this project, but we don&rsquo;t want to be
+distracted by re-implementing all of twitter.commons in order to
+create a perfect Aurora client.</li>
+</ul>
+
+<h2 id="background">Background</h2>
+
+<p>Aurora is a system that&rsquo;s used to run and manage services and
+service-like jobs running in a datacenter. Aurora takes care of
+allocating resources in order to schedule and run jobs without
+requiring teams to manage dedicated hardware. The heart of Aurora is
+called the scheduler, and is responsible for finding and assigning
+resources to tasks.</p>
+
+<p>The Aurora scheduler provides a thrift API. The scheduler API is
+low-level and difficult to interact with. Users do not interact
+directly with the Aurora API; instead, they use a command-line tool,
+which provides a collection of easy-to-use commands. This command-line
+tool, in turn, talks to the scheduler API to launch and manage jobs in
+datacenter clusters. The command-line tool is called the Aurora
+client.</p>
+
+<p>The current implementation of the Aurora client is haphazard,
+and really needs to be cleaned up:</p>
+
+<ul>
+<li>The code is monolithic and hard to maintain. It&rsquo;s implemented using
+<code>twitter.common.app</code>, which assumes that all of the command code lives
+in a single source file. To work around this, and allow some
+subdivision, it uses a hack of <code>twitter.common.app</code> to force
+registration of commands from multiple modules. It&rsquo;s hard to
+understand, and hard to modify.</li>
+<li>The current code is very difficult to test. Because of the way it&rsquo;s
+built, there is no consistent way of passing key application data
+around. As a result, each unit test of client operations needs a
+difficult-to-assemble custom setup of mock objects.</li>
+<li>The current code handles errors poorly, and it is difficult to
+fix. Many common errors produce unacceptable results. For example,
+issuing an unknown command generates an error message &ldquo;main takes 0
+parameters but received 1&rdquo;; passing an invalid parameter to other
+commands frequently produces a stack trace.</li>
+<li>The current command line is confusing for users. There are several
+different kinds of objects manipulated by the Aurora command line,
+and the difference between them is often not entirely clear. (What&rsquo;s
+the difference between a job and a configuration?)
+For each type of object, there are different
+commands, and it&rsquo;s frequently not clear just which command should be
+used for which object.</li>
+</ul>
+
+<p>Instead of continuing to let it develop and evolve randomly, it&rsquo;s time
+to take a principled look at the Aurora command line, and figure out
+how to make command line processing make sense. At the same time, the
+code needs to be cleaned up, and divided into small comprehensible
+units based on a plugin architecture.</p>
+
+<h2 id="requirements">Requirements</h2>
+
+<p>Aurora is aimed at engineers who run jobs and services in a
+datacenter. As a result, the requirements for the aurora client are
+all engineering focused:</p>
+
+<ul>
+<li><strong>Consistency</strong>: commands should follow a consistent structure, so that
+users can apply knowledge and intuition gained from working with
+some aurora commands to new commands. This means that when commands
+can re-use the same options, they should; that objects should be
+referred to by consistent syntax throughout the tool.</li>
+<li><strong>Helpfulness</strong>: commands should be structured so that the system can
+generate helpful error messages. If a user just runs &ldquo;aurora&rdquo;, they
+should get a basic usage message. If they try to run an invalid
+command, they should get a message that the command is invalid, not
+a stack dump or &ldquo;command main() takes 0 parameters but received
+2&rdquo;. Commands should not generate extraneous output that obscures the
+key facts that the user needs to know, and the default behavior of
+commands should not generate outputs that will be routinely ignored
+by users.</li>
+<li><strong>Extensibility</strong>: it should be easy to plug in new commands,
+including custom commands, to adapt the Aurora client to new
+environments.</li>
+<li><strong>Script-friendly command output</strong>: every command should at least include
+an option that generates output that&rsquo;s script-friendly. Scripts should be
+able to work with command-output without needing to do screen scraping.</li>
+<li><strong>Scalability</strong>: the tools should be usable for any foreseeable size
+of Aurora datacenters and machine clusters.</li>
+</ul>
+
+<h2 id="design-overview">Design Overview</h2>
+
+<p>The Aurora client will be reimplemented using a noun-verb model,
+similar to the cmdlet model used by Monad/Windows Powershell. Users
+will work by providing a noun for the type of object being operated
+on, and a verb for the specific operation being performed on the
+object, followed by parameters. For example, to create a job, the user
+would execute: &ldquo;<code>aurora job create smfd/mchucarroll/devel/jobname
+job.aurora</code>&rdquo;. The noun is <code>job</code> and the verb is <code>create</code>.</p>
+
+<p>The client will be implemented following that noun-verb
+convention. Each noun will be a separate component, which can be
+registered into the command-line framework. Each verb will be
+implemented by a class that registers with the appropriate noun. Nouns
+and verbs will each provide methods that add their command line
+options and parameters to the options parser, using the Python
+argparse library.</p>
+
+<h2 id="detailed-design">Detailed Design</h2>
+
+<h3 id="interface">Interface</h3>
+
+<p>In this section, we&rsquo;ll walk through the types of objects that the
+client can manipulate, and the operations that need to be provided for
+each object. These form the primary interface that engineers will use
+to interact with Aurora.</p>
+
+<p>In the command-line, each of the object types will have an Aurora
+subcommand. The commands to manipulate the object type will follow the
+type. For example, here are several commands in the old syntax
+contrasted against the new noun/verb syntax.</p>
+
+<ul>
+<li>Get quota for a role:
+
+<ul>
+<li>Noun/Verb syntax:  <code>aurora quota get west/www-data</code></li>
+<li>Old syntax: <code>aurora get_quota --cluster=smf1 www-data</code></li>
+</ul></li>
+<li>Create job:
+
+<ul>
+<li>Noun/Verb syntax: <code>aurora job create west/www-data/test/job job.aurora</code></li>
+<li>Old syntax: <code>aurora create west/www-data/test/job job.aurora</code></li>
+</ul></li>
+<li>Schedule a job to run at a specific interval:
+
+<ul>
+<li>Noun/verb: <code>aurora cron schedule east/www-data/test/job job.aurora</code></li>
+<li>Old: <code>aurora create east/www-data/test/job job.aurora</code></li>
+</ul></li>
+</ul>
+
+<p>As you can see in these examples, the new syntax is more consistent:
+you always specify the cluster where a command executes as part of an
+identifier, where in the old syntax, it was sometimes part of the
+jobkey and sometimes specified with a &ldquo;&ndash;cluster&rdquo; option.</p>
+
+<p>The new syntax is also more clear and explicit: even without knowing
+much about Aurora, it&rsquo;s clear what objects each command is acting on,
+where in the old syntax, commands like &ldquo;create&rdquo; are unclear.</p>
+
+<h3 id="the-job-noun">The Job Noun</h3>
+
+<p>A job is a configured program ready to run in Aurora. A job is,
+conceptually, a task factory: when a job is submitted to the Aurora
+scheduler, it creates a collection of tasks. The job contains a
+complete description of everything it needs to create a collection of
+tasks. (Note that this subsumes &ldquo;service&rdquo; commands. A service is just
+a task whose configuration sets the is_service flag, so we don&rsquo;t have
+separate commands for working with services.) Jobs are specified using
+<code>cluster/role/env/name</code> jobkey syntax.</p>
+
+<ul>
+<li><code>aurora job create *jobkey* *config*</code>:  submits a job to a cluster, launching the task(s) specified by the job config.</li>
+<li><code>aurora job status *jobkey*</code>: query job status. Prints information about the job,
+whether it&rsquo;s running, etc., to standard out. If jobkey includes
+globs, it should list all jobs that match the glob</li>
+<li><code>aurora job kill *jobkey*/*instanceids*</code>: kill/stop some of a jobs instances. This stops a job&#39; tasks; if the job
+has service tasks, they&rsquo;ll be  disabled, so that they won&rsquo;t restart.</li>
+<li><code>aurora job killall *jobkey*</code>: kill all of the instances of a job. This
+is distinct from the <em>kill</em> command as a safety measure: omitting the
+instances from a kill command shouldn&rsquo;t result in destroying the entire job.</li>
+<li><code>aurora job restart *jobkey*</code>: conceptually, this will kill a job, and then
+launch it again. If the job does not exist, then fail with an error
+message.  In fact, the underlying implementation does the
+kill/relaunch on a rolling basis - so it&rsquo;s not an immediate kill of
+all shards/instances, followed by a delay as all instances relaunch,
+but rather a controlled gradual process.</li>
+<li><code>aurora job list *jobkey*</code>: list all jobs that match the jobkey spec that are
+registered with the scheduler. This will include both jobs that are
+currently running, and jobs that are scheduled to run at a later
+time. The job key can be partial: if it specifies cluster, all jobs
+on the cluster will be listed; cluster/role, all jobs running on the cluster under the role will be listed, etc.</li>
+</ul>
+
+<h2 id="the-schedule-noun-(cron)">The Schedule Noun (Cron)</h2>
+
+<p>Note (3/21/2014): The &ldquo;cron&rdquo; noun is <em>not</em> implemented yet.</p>
+
+<p>Cron is a scheduler adjunct that periodically runs a job on a
+schedule. The cron commands all manipulate cron schedule entries. The
+schedules are specified as a part of the job configuration.</p>
+
+<ul>
+<li><code>aurora cron schedule jobkey config</code>: schedule a job to run by cron.</li>
+<li><code>aurora cron deschedule jobkey</code>: removes a jobs entry from the cron schedule.</li>
+<li><code>aurora cron status jobkey</code>: query for a scheduled job&rsquo;s status.</li>
+</ul>
+
+<h2 id="the-quota-noun">The Quota Noun</h2>
+
+<p>A quota is a data object maintained by the scheduler that specifies the maximum
+resources that may be consumed by jobs owned by a particular role. In the future,
+we may add new quota types. At some point, we&rsquo;ll also probably add an administrators
+command to set quotas.</p>
+
+<ul>
+<li><code>aurora quota get *cluster/role*</code></li>
+</ul>
+
+<h2 id="implementation">Implementation</h2>
+
+<p>The current command line is monolithic. Every command on an Aurora
+object is a top-level command in the Aurora client. In the
+restructured command line, each of the primary object types
+manipulated by Aurora should have its own sub-command.</p>
+
+<ul>
+<li>Advantages of this approach:
+
+<ul>
+<li>Easier to detangle the command-line processing. The top-level
+command-processing will be a small set of subcommand
+processors. Option processing for each subcommand can be offloaded
+to a separate module.</li>
+<li>The aurora top-level help command will be much more
+comprehensible. Instead of giving a huge list of every possible
+command, it will present the list of top-level object types, and
+then users can request help on the commands for a specific type
+of object.</li>
+<li>The sub-commands can be separated into distinct command-line
+tools when appropriate.</li>
+</ul></li>
+</ul>
+
+<h3 id="command-structure-and-options-processing">Command Structure and Options Processing</h3>
+
+<p>The implementation will follow closely on Pants goals. Pants goals use
+a static registration system to add new subcommands. In pants, each
+goal command is an implementation of a command interface, and provides
+implementations of methods to register options and parameters, and to
+actually execute the command. In this design, commands are modular and
+easy to implement, debug, and combine in different ways.</p>
+
+<p>For the Aurora client, we plan to use a two-level variation of the
+basic concept from pants. At the top-level we will have nouns. A noun
+will define some common command-line parameters required by all of its
+verbs, and will provide a registration hook for attaching verbs. Nouns
+will be implemented as a subclass of a basic Noun type.</p>
+
+<p>Each verb will, similarly, be implemented as a subclass of Verb. Verbs
+will be able to specify command-line options and parameters.</p>
+
+<p>Both <code>Noun</code> and <code>Verb</code> will be subclasses of a common base-class <code>AuroraCommand</code>:</p>
+<pre class="highlight text">class AuroraCommand(object):
+  def get_options(self):
+  &quot;&quot;&quot;Gets the set of command-line options objects for this command.
+  The result is a list of CommandOption objects.
+   &quot;&quot;&quot;
+    pass
+
+  @property
+  def help(self):
+    &quot;&quot;&quot;Returns the help message for this command&quot;&quot;&quot;
+
+  @property
+  def usage(self):
+    &quot;&quot;&quot;Returns a short usage description of the command&quot;&quot;&quot;
+
+  @property
+  def name(self):
+    &quot;&quot;&quot;Returns the command name&quot;&quot;&quot;
+</pre>
+<p>A command-line tool will be implemented as an instance of a <code>CommandLine</code>:</p>
+<pre class="highlight text">class CommandLine(object):
+  &quot;&quot;&quot;The top-level object implementing a command-line application.&quot;&quot;&quot;
+
+  @property
+  def name(self):
+    &quot;&quot;&quot;Returns the name of this command-line tool&quot;&quot;&quot;
+
+  def print_out(self, str):
+    print(str)
+
+  def print_err(self, str):
+    print(str, file=sys.stderr)
+
+  def register_noun(self, noun):
+    &quot;&quot;&quot;Adds a noun to the application&quot;&quot;&quot;
+
+  def register_plugin(self, plugin):
+     &quot;&quot;&quot;Adds a configuration plugin to the system&quot;&quot;&quot;
+</pre>
+<p>Nouns are registered into a command-line using the <code>register_noun</code>
+method. They are weakly coupled to the application, making it easy to
+use a single noun in several different command-line tools. Nouns allow
+the registration of verbs using the <code>register_verb</code> method.</p>
+
+<p>When commands execute, they&rsquo;re given an instance of a <em>context object</em>.
+The context object must be an instance of a subclass of <code>AuroraCommandContext</code>.
+Options, parameters, and IO are all accessed using the context object. The context
+is created dynamically by the noun object owning the verb being executed. Developers
+are strongly encouraged to implement custom contexts for their nouns, and move functionality
+shared by the noun&rsquo;s verbs into the context object. The context interface is:</p>
+<pre class="highlight text">class Context(object):
+  class Error(Exception): pass
+
+  class ArgumentException(Error): pass
+
+  class CommandError(Error):
+
+  @classmethod
+  def exit(cls, code, msg):
+    &quot;&quot;&quot;Exit the application with an error message&quot;&quot;&quot;
+    raise cls.CommandError(code, msg)
+
+ def print_out(self, msg, indent=0):
+   &quot;&quot;&quot;Prints a message to standard out, with an indent&quot;&quot;&quot;
+
+ def print_err(self, msg, indent=0):
+   &quot;&quot;&quot;Prints a message to standard err, with an indent&quot;&quot;&quot;
+</pre>
+<p>In addition to nouns and verbs, there&rsquo;s one more kind of registerable
+component, called a <em>configuration plugin</em>. These objects add a set of
+command-line options that can be passed to <em>all</em> of the commands
+implemented in the tool. Before the command is executed, the
+configuration plugin will be invoked, and will process its
+command-line arguments. This is useful for general configuration
+changes, like establish a secure tunnel to talk to machines in a
+datacenter. (A useful way to think of a plugin is as something like an
+aspect that can be woven in to aurora to provide environment-specific
+configuration.) A configuration plugin is implemented as an instance
+of class <code>ConfigurationPlugin</code>, and registered with the
+<code>register_plugin</code> method of the <code>CommandLine</code> object. The interface of
+a plugin is:</p>
+<pre class="highlight text">class ConfigurationPlugin(object):
+  &quot;&quot;&quot;A component that can be plugged in to a command-line.&quot;&quot;&quot;
+
+  @abstractmethod
+  def get_options(self):
+    &quot;&quot;&quot;Return the set of options processed by this plugin&quot;&quot;&quot;
+
+  @abstractmethod
+  def execute(self, context):
+    &quot;&quot;&quot;Run the context/command line initialization code for this plugin.&quot;&quot;&quot;
+</pre>
+<h3 id="command-execution">Command Execution</h3>
+
+<p>The options process and command execution is built as a facade over Python&rsquo;s
+standard argparse. All of the actual argument processing is done by the
+argparse library.</p>
+
+<p>Once the options are processed, the framework will start to execute the command. Command execution consists of:</p>
+
+<h1 id="create-a-context-object.-the-framework-will-use-the-argparse-options-to-identify">Create a context object. The framework will use the argparse options to identify</h1>
+
+<p>which noun is being invoked, and will call that noun&rsquo;s <code>create_context</code> method.
+  The argparse options object will be stored in the context.</p>
+
+<h1 id="execute-any-configuration-plugins.-before-any-command-is-invoked,-the-framework">Execute any configuration plugins. Before any command is invoked, the framework</h1>
+
+<p>will first iterate over all of the registered configuration plugins. For each
+  plugin, it will invoke the <code>execute</code> method.</p>
+
+<h1 id="the-noun-will-use-the-context-to-find-out-what-verb-is-being-invoked,-and-it-will">The noun will use the context to find out what verb is being invoked, and it will</h1>
+
+<p>then call that verb&rsquo;s <code>execute</code> method.</p>
+
+<h1 id="the-command-will-exit.-its-return-code-will-be-whatever-was-returned-by-the-verb&#39;s">The command will exit. Its return code will be whatever was returned by the verb&rsquo;s</h1>
+
+<p><code>execute</code> method.</p>
+
+<p>Commands are expected to return a code from a list of standard exit codes,
+which can be found in <code>src/main/python/apache/aurora/client/cli/__init__.py</code>.</p>
+
+	  </div>
+      <div class="container">
+    <hr>
+    <footer class="footer">
+        <div class="row-fluid">
+            <div class="span2 text-left">
+                <h3>Links</h3>
+                <ul class="unstyled">
+                    <li><a href="/downloads/">Downloads</a></li>
+                    <li><a href="/developers/">Developers</a></li>                    
+                </ul>
+            </div>
+            <div class="span3 text-left">
+                <h3>Community</h3>
+                <ul class="unstyled">
+                    <li><a href="/community/">Mailing Lists</a></li>
+                    <li><a href="http://issues.apache.org/jira/browse/aurora">Issue Tracking</a></li>
+                    <li><a href="/docs/howtocontribute/">How To Contribute</a></li>
+                </ul>
+            </div>
+            <div class="span7 text-left">
+            	<h3>Apache Software Foundation</h3>
+
+							<div class="span8">
+                Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. Apache, Apache Thrift, and the Apache feather logo are trademarks of The Apache Software Foundation. Currently part of the <a href="http://incubator.apache.org">Apache Incubator</a>.
+							</div>
+							<div class=" pull-right">
+								<a href="http://incubator.apache.org" class="logo"><img src="/assets/img/apache_incubator_logo.png" alt="Apache Incubator" class="pull-right"/></a>
+							</div>
+            </div>
+
+        </div>
+
+    </footer>
+</div>
+
+	</body>
+</html>
+

Added: incubator/aurora/site/publish/documentation/latest/committers/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/committers/index.html?rev=1623604&view=auto
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/committers/index.html (added)
+++ incubator/aurora/site/publish/documentation/latest/committers/index.html Tue Sep  9 00:47:19 2014
@@ -0,0 +1,151 @@
+<html>
+    <head>
+        <meta charset="utf-8">
+        <title>Apache Aurora</title>
+		    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+		    <meta name="description" content="">
+		    <meta name="author" content="">
+
+		    <link href="/assets/css/bootstrap.css" rel="stylesheet">
+		    <link href="/assets/css/bootstrap-responsive.min.css" rel="stylesheet">
+		    <link href="/assets/css/main.css" rel="stylesheet">
+				
+		    <!-- JS -->
+		    <script type="text/javascript" src="/assets/js/jquery-1.10.1.min.js"></script>
+		    <script type="text/javascript" src="/assets/js/bootstrap-dropdown.js"></script>
+		
+				<!-- Analytics -->
+				<script type="text/javascript">
+					  var _gaq = _gaq || [];
+					  _gaq.push(['_setAccount', 'UA-45879646-1']);
+					  _gaq.push(['_setDomainName', 'apache.org']);
+					  _gaq.push(['_trackPageview']);
+
+					  (function() {
+					    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+					    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+					    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+					  })();
+				</script>
+	</head>
+    <body>	
+      <div class="navbar navbar-static-top">
+  <div class="navbar-inner">
+    <div class="container">
+	    <a href="/" class="logo"><img src="/assets/img/aurora_logo.png" alt="Apache Aurora logo" /></a>
+      <ul class="nav">
+				<li><a href="/documentation/latest/">Documentation</a></li>
+        <li><a href="/downloads/">Download</a></li>
+        <li><a href="/community">Community</a></li>
+      </ul>
+    </div>
+  </div>
+</div>
+
+<div class="container">
+<!-- magical breadcrumbs -->
+<ul class="breadcrumb">
+  <li>
+    <div class="dropdown">
+      <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a>
+      <ul class="dropdown-menu" role="menu">
+        <li><a href="http://www.apache.org">Apache Homepage</a></li>
+        <li><a href="http://www.apache.org/licenses/">Apache License</a></li>
+        <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>  
+        <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+        <li><a href="http://www.apache.org/security/">Security</a></li>
+      </ul>
+    </div>
+  </li>
+  <li><span class="divider">&bull;</span></li>
+  <li><a href="http://incubator.apache.org">Apache Incubator</a></li>
+  <li><span class="divider">&bull;</span></li>
+  <li><a href="http://aurora.incubator.apache.org">Apache Aurora</a></li>
+</ul>
+<!-- /breadcrumb -->
+	
+      <div class="container">
+        <h2 id="setting-up-your-email-account">Setting up your email account</h2>
+
+<p>Once your Apache ID has been set up you can configure your account and add ssh keys and setup an
+email forwarding address at</p>
+
+<p><a href="http://id.apache.org">http://id.apache.org</a></p>
+
+<p>Additional instructions for setting up your new committer email can be found at</p>
+
+<p><a href="http://www.apache.org/dev/user-email.html">http://www.apache.org/dev/user-email.html</a></p>
+
+<p>The recommended setup is to configure all services (mailing lists, JIRA, ReviewBoard) to send
+emails to your @apache.org email address.</p>
+
+<h2 id="creating-a-release">Creating a release</h2>
+
+<p>The following will guide you through the steps to create a release candidate, vote, and finally an
+official Apache Aurora release. Before starting your gpg key should be in the KEYS file and you
+must have access to commit to the dist.a.o repositories.</p>
+
+<ol>
+<li><p>Ensure that all issues resolved for this release candidate are tagged with the correct Fix
+Version in Jira, the changelog script will use this to generate the CHANGELOG in step #2.</p></li>
+<li><p>Create a release candidate. This will automatically update the CHANGELOG and commit it, create a
+branch and update the current version within the trunk. To create a minor version update and publish
+it run</p>
+<pre class="highlight text">       ./build-support/release/release-candidate -l m -p
+</pre></li>
+<li><p>Update, if necessary, the draft email created from the <code>release-candidate</code> script in step #2 and
+send the [VOTE] email to the dev@ and private@ mailing lists. You can verify the release signature
+and checksums by running</p>
+<pre class="highlight text">        ./build-support/release/verify-release-candidate
+</pre></li>
+<li><p>Wait for the vote to complete. If the vote fails address any issues and go back to step #1 and
+run again, this time you will use the -r flag to increment the release candidate version. This will
+automatically clean up the release candidate rc0 branch and source distribution.</p>
+<pre class="highlight text">       ./build-support/release/release-candidate -l m -r 1 -p
+</pre></li>
+<li><p>Once the vote has successfully passed create the release</p>
+<pre class="highlight text">       ./build-support/release/release
+</pre></li>
+<li><p>Update the draft email created fom the <code>release</code> script in step #5 to include the Apache ID&rsquo;s for
+all binding votes and send the [RESULT][VOTE] email to the dev@ and private@ mailing lists.</p></li>
+</ol>
+
+	  </div>
+      <div class="container">
+    <hr>
+    <footer class="footer">
+        <div class="row-fluid">
+            <div class="span2 text-left">
+                <h3>Links</h3>
+                <ul class="unstyled">
+                    <li><a href="/downloads/">Downloads</a></li>
+                    <li><a href="/developers/">Developers</a></li>                    
+                </ul>
+            </div>
+            <div class="span3 text-left">
+                <h3>Community</h3>
+                <ul class="unstyled">
+                    <li><a href="/community/">Mailing Lists</a></li>
+                    <li><a href="http://issues.apache.org/jira/browse/aurora">Issue Tracking</a></li>
+                    <li><a href="/docs/howtocontribute/">How To Contribute</a></li>
+                </ul>
+            </div>
+            <div class="span7 text-left">
+            	<h3>Apache Software Foundation</h3>
+
+							<div class="span8">
+                Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. Apache, Apache Thrift, and the Apache feather logo are trademarks of The Apache Software Foundation. Currently part of the <a href="http://incubator.apache.org">Apache Incubator</a>.
+							</div>
+							<div class=" pull-right">
+								<a href="http://incubator.apache.org" class="logo"><img src="/assets/img/apache_incubator_logo.png" alt="Apache Incubator" class="pull-right"/></a>
+							</div>
+            </div>
+
+        </div>
+
+    </footer>
+</div>
+
+	</body>
+</html>
+

Modified: incubator/aurora/site/publish/documentation/latest/configuration-reference/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/configuration-reference/index.html?rev=1623604&r1=1623603&r2=1623604&view=diff
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/configuration-reference/index.html (original)
+++ incubator/aurora/site/publish/documentation/latest/configuration-reference/index.html Tue Sep  9 00:47:19 2014
@@ -109,6 +109,7 @@
 <li><a href="#services">Services</a></li>
 <li><a href="#updateconfig-objects">UpdateConfig Objects</a></li>
 <li><a href="#healthcheckconfig-objects">HealthCheckConfig Objects</a></li>
+<li><a href="#announcer-objects">Announcer Objects</a></li>
 </ul></li>
 <li><a href="#specifying-scheduling-constraints">Specifying Scheduling Constraints</a></li>
 <li><a href="#template-namespaces">Template Namespaces</a>
@@ -611,7 +612,7 @@ either due to human error or machine fai
 <tr>
 <td><code>watch_secs</code></td>
 <td style="text-align: center">Integer</td>
-<td>Minimum number of seconds a shard must remain in <code>RUNNING</code> state before considered a success (Default: 30)</td>
+<td>Minimum number of seconds a shard must remain in <code>RUNNING</code> state before considered a success (Default: 45)</td>
 </tr>
 <tr>
 <td><code>max_per_shard_failures</code></td>
@@ -639,12 +640,12 @@ either due to human error or machine fai
 <tr>
 <td><code>initial_interval_secs</code></td>
 <td style="text-align: center">Integer</td>
-<td>Initial delay for performing an HTTP health check. (Default: 60)</td>
+<td>Initial delay for performing an HTTP health check. (Default: 15)</td>
 </tr>
 <tr>
 <td><code>interval_secs</code></td>
 <td style="text-align: center">Integer</td>
-<td>Interval on which to check the task&rsquo;s health via HTTP. (Default: 30)</td>
+<td>Interval on which to check the task&rsquo;s health via HTTP. (Default: 10)</td>
 </tr>
 <tr>
 <td><code>timeout_secs</code></td>
@@ -658,6 +659,56 @@ either due to human error or machine fai
 </tr>
 </tbody></table>
 
+<h3 id="announcer-objects">Announcer Objects</h3>
+
+<p>If the <code>announce</code> field in the Job configuration is set, each task will be
+registered in the ServerSet <code>/aurora/role/environment/jobname</code> in the
+zookeeper ensemble configured by the executor.  If no Announcer object is specified,
+no announcement will take place.  For more information about ServerSets, see the <a href="/documentation/latest/user-guide/">User Guide</a>.</p>
+
+<table><thead>
+<tr>
+<th>object</th>
+<th style="text-align: center">type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>primary_port</code></td>
+<td style="text-align: center">String</td>
+<td>Which named port to register as the primary endpoint in the ServerSet (Default: <code>http</code>)</td>
+</tr>
+<tr>
+<td><code>portmap</code></td>
+<td style="text-align: center">dict</td>
+<td>A mapping of additional endpoints to announced in the ServerSet (Default: <code>{ &#39;aurora&#39;: &#39;{{primary_port}}&#39; }</code>)</td>
+</tr>
+</tbody></table>
+
+<h3 id="port-aliasing-with-the-announcer-portmap">Port aliasing with the Announcer <code>portmap</code></h3>
+
+<p>The primary endpoint registered in the ServerSet is the one allocated to the port
+specified by the <code>primary_port</code> in the <code>Announcer</code> object, by default
+the <code>http</code> port.  This port can be referenced from anywhere within a configuration
+as <code>{{thermos.ports[http]}}</code>.</p>
+
+<p>Without the port map, each named port would be allocated a unique port number.
+The <code>portmap</code> allows two different named ports to be aliased together.  The default
+<code>portmap</code> aliases the <code>aurora</code> port (i.e. <code>{{thermos.ports[aurora]}}</code>) to
+the <code>http</code> port.  Even though the two ports can be referenced independently,
+only one port is allocated by Mesos.  Any port referenced in a <code>Process</code> object
+but which is not in the portmap will be allocated dynamically by Mesos and announced as well.</p>
+
+<p>It is possible to use the portmap to alias names to static port numbers, e.g.
+<code>{&#39;http&#39;: 80, &#39;https&#39;: 443, &#39;aurora&#39;: &#39;http&#39;}</code>.  In this case, referencing
+<code>{{thermos.ports[aurora]}}</code> would look up <code>{{thermos.ports[http]}}</code> then
+find a static port 80.  No port would be requested of or allocated by Mesos.</p>
+
+<p>Static ports should be used cautiously as Aurora does nothing to prevent two
+tasks with the same static port allocations from being co-scheduled.
+External constraints such as slave attributes should be used to enforce such
+guarantees should they be needed.</p>
+
 <h1 id="specifying-scheduling-constraints">Specifying Scheduling Constraints</h1>
 
 <p>Most users will not need to specify constraints explicitly, as the

Modified: incubator/aurora/site/publish/documentation/latest/configuration-tutorial/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/configuration-tutorial/index.html?rev=1623604&r1=1623603&r2=1623604&view=diff
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/configuration-tutorial/index.html (original)
+++ incubator/aurora/site/publish/documentation/latest/configuration-tutorial/index.html Tue Sep  9 00:47:19 2014
@@ -650,7 +650,7 @@ maximum number of shards to update in on
 <li>  <code>restart_threshold</code>: An integer, defaulting to <code>60</code>, specifying
 the maximum number of seconds before a shard must move into the
 <code>RUNNING</code> state before considered a failure.</li>
-<li>  <code>watch_secs</code>: An integer, defaulting to <code>30</code>, specifying the
+<li>  <code>watch_secs</code>: An integer, defaulting to <code>45</code>, specifying the
 minimum number of seconds a shard must remain in the <code>RUNNING</code>
 state before considered a success.</li>
 <li>  <code>max_per_shard_failures</code>: An integer, defaulting to <code>0</code>,
@@ -668,9 +668,9 @@ used if a health port was assigned with 
 <code>HealthCheckConfig</code> parameters are:
 
 <ul>
-<li>  <code>initial_interval_secs</code>: An integer, defaulting to <code>60</code>,
+<li>  <code>initial_interval_secs</code>: An integer, defaulting to <code>15</code>,
 specifying the initial delay for doing an HTTP health check.</li>
-<li>  <code>interval_secs</code>: An integer, defaulting to <code>30</code>, specifying the
+<li>  <code>interval_secs</code>: An integer, defaulting to <code>10</code>, specifying the
 number of seconds in the interval between checking the Task&rsquo;s
 health.</li>
 <li>  <code>timeout_secs</code>: An integer, defaulting to <code>1</code>, specifying the

Modified: incubator/aurora/site/publish/documentation/latest/contributing/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/contributing/index.html?rev=1623604&r1=1623603&r2=1623604&view=diff
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/contributing/index.html (original)
+++ incubator/aurora/site/publish/documentation/latest/contributing/index.html Tue Sep  9 00:47:19 2014
@@ -65,23 +65,16 @@
 <!-- /breadcrumb -->
 	
       <div class="container">
-        <h2 id="getting-your-reviewboard-account">Getting your ReviewBoard Account</h2>
+        <h2 id="find-something-to-do">Find Something to Do</h2>
 
-<p>Go to <a href="https://reviews.apache.org">https://reviews.apache.org</a> and create an account.</p>
-
-<h2 id="setting-up-your-email-account-(committers)">Setting up your email account (committers)</h2>
-
-<p>Once your Apache ID has been set up you can configure your account and add ssh keys and
-setup an email forwarding address at</p>
-
-<p><a href="http://id.apache.org">http://id.apache.org</a></p>
-
-<p>Additional instructions for setting up your new committer email can be found at</p>
+<p>There are issues in <a href="https://issues.apache.org/jira/browse/AURORA">Jira</a> with the
+<a href="https://issues.apache.org/jira/browse/AURORA-189?jql=project%20%3D%20AURORA%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20newbie%20ORDER%20BY%20priority%20DESC">&ldquo;newbie&rdquo; tag</a>
+that are good starting places for new Aurora contributors; pick one of these and dive in! Once
+you&rsquo;ve got a patch, the next step is to post a review.</p>
 
-<p><a href="http://www.apache.org/dev/user-email.html">http://www.apache.org/dev/user-email.html</a></p>
+<h2 id="getting-your-reviewboard-account">Getting your ReviewBoard Account</h2>
 
-<p>The recommended setup is to configure all services (mailing lists, JIRA, ReviewBoard) to
-send emails to your @apache.org email address.</p>
+<p>Go to <a href="https://reviews.apache.org">https://reviews.apache.org</a> and create an account.</p>
 
 <h2 id="setting-up-your-reviewboard-environment">Setting up your ReviewBoard Environment</h2>
 
@@ -91,8 +84,10 @@ Subsequent runs will cache your login cr
 <h2 id="submitting-a-patch-for-review">Submitting a Patch for Review</h2>
 
 <p>Post a review with <code>rbt</code>, fill out the fields in your browser and hit Publish.</p>
-<pre class="highlight text">./rbt post -o -g
+<pre class="highlight text">./rbt post -o
 </pre>
+<p>Once you&rsquo;ve done this, you probably want to mark the associated Jira issue as Reviewable.</p>
+
 <h2 id="updating-an-existing-review">Updating an Existing Review</h2>
 
 <p>Incorporate review feedback, make some more commits, update your existing review, fill out the
@@ -123,6 +118,12 @@ git pull origin master
 git show master  # Verify everything looks sane, author is correct
 git push origin master
 </pre>
+<h2 id="cleaning-up">Cleaning Up</h2>
+
+<p>Your patch has landed, congratulations! The last thing you&rsquo;ll want to do before moving on to your
+next fix is to clean up your Jira and Reviewboard. The former of which should be marked as
+&ldquo;Resolved&rdquo; while the latter should be marked as &ldquo;Submitted&rdquo;.</p>
+
 	  </div>
       <div class="container">
     <hr>

Modified: incubator/aurora/site/publish/documentation/latest/deploying-aurora-scheduler/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/deploying-aurora-scheduler/index.html?rev=1623604&r1=1623603&r2=1623604&view=diff
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/deploying-aurora-scheduler/index.html (original)
+++ incubator/aurora/site/publish/documentation/latest/deploying-aurora-scheduler/index.html Tue Sep  9 00:47:19 2014
@@ -72,7 +72,7 @@ old jobs.</p>
 
 <p>Aurora is a standalone Java server. As part of the build process it creates a bundle of all its
 dependencies, with the notable exceptions of the JVM and libmesos. Each target server should have
-a JVM (Java 7 or higher) and libmesos (0.17.0) installed.</p>
+a JVM (Java 7 or higher) and libmesos (0.18.0) installed.</p>
 
 <h2 id="creating-the-distribution-.zip-file-(optional)">Creating the Distribution .zip File (Optional)</h2>
 
@@ -110,7 +110,6 @@ sudo ln -nfs &quot;$(ls -dt /usr/local/a
 <span class="c"># Flags controlling the scheduler.</span>
 <span class="nv">AURORA_FLAGS</span><span class="o">=(</span>
   -http_port<span class="o">=</span>8081
-  -thrift_port<span class="o">=</span>8082
   <span class="c"># Log configuration, etc.</span>
 <span class="o">)</span>
 
@@ -171,15 +170,14 @@ should be set to <code>2</code>, and in 
 
 <h2 id="network-considerations">Network considerations</h2>
 
-<p>The Aurora scheduler listens on 3 ports - a Thrift port for client RPCs, an admin web UI, and a
-libprocess (HTTP+Protobuf) port used to communicate with the Mesos master and for the log
+<p>The Aurora scheduler listens on 2 ports - an HTTP port used for client RPCs and a web UI,
+and a libprocess (HTTP+Protobuf) port used to communicate with the Mesos master and for the log
 replication protocol. These can be left unconfigured (the scheduler publishes all selected ports
 to ZooKeeper) or explicitly set in the startup script as follows:</p>
 <pre class="highlight text"># ...
 AURORA_FLAGS=(
   # ...
   -http_port=8081
-  -thrift_port=8082
   # ...
 )
 # ...

Added: incubator/aurora/site/publish/documentation/latest/developing-aurora-client/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/developing-aurora-client/index.html?rev=1623604&view=auto
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/developing-aurora-client/index.html (added)
+++ incubator/aurora/site/publish/documentation/latest/developing-aurora-client/index.html Tue Sep  9 00:47:19 2014
@@ -0,0 +1,230 @@
+<html>
+    <head>
+        <meta charset="utf-8">
+        <title>Apache Aurora</title>
+		    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+		    <meta name="description" content="">
+		    <meta name="author" content="">
+
+		    <link href="/assets/css/bootstrap.css" rel="stylesheet">
+		    <link href="/assets/css/bootstrap-responsive.min.css" rel="stylesheet">
+		    <link href="/assets/css/main.css" rel="stylesheet">
+				
+		    <!-- JS -->
+		    <script type="text/javascript" src="/assets/js/jquery-1.10.1.min.js"></script>
+		    <script type="text/javascript" src="/assets/js/bootstrap-dropdown.js"></script>
+		
+				<!-- Analytics -->
+				<script type="text/javascript">
+					  var _gaq = _gaq || [];
+					  _gaq.push(['_setAccount', 'UA-45879646-1']);
+					  _gaq.push(['_setDomainName', 'apache.org']);
+					  _gaq.push(['_trackPageview']);
+
+					  (function() {
+					    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+					    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+					    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+					  })();
+				</script>
+	</head>
+    <body>	
+      <div class="navbar navbar-static-top">
+  <div class="navbar-inner">
+    <div class="container">
+	    <a href="/" class="logo"><img src="/assets/img/aurora_logo.png" alt="Apache Aurora logo" /></a>
+      <ul class="nav">
+				<li><a href="/documentation/latest/">Documentation</a></li>
+        <li><a href="/downloads/">Download</a></li>
+        <li><a href="/community">Community</a></li>
+      </ul>
+    </div>
+  </div>
+</div>
+
+<div class="container">
+<!-- magical breadcrumbs -->
+<ul class="breadcrumb">
+  <li>
+    <div class="dropdown">
+      <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a>
+      <ul class="dropdown-menu" role="menu">
+        <li><a href="http://www.apache.org">Apache Homepage</a></li>
+        <li><a href="http://www.apache.org/licenses/">Apache License</a></li>
+        <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>  
+        <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+        <li><a href="http://www.apache.org/security/">Security</a></li>
+      </ul>
+    </div>
+  </li>
+  <li><span class="divider">&bull;</span></li>
+  <li><a href="http://incubator.apache.org">Apache Incubator</a></li>
+  <li><span class="divider">&bull;</span></li>
+  <li><a href="http://aurora.incubator.apache.org">Apache Aurora</a></li>
+</ul>
+<!-- /breadcrumb -->
+	
+      <div class="container">
+        <h1 id="getting-started">Getting Started</h1>
+
+<p>Aurora consists of four main pieces: the scheduler (which finds resources in the cluster that can be used to run a job), the executor (which uses the resources assigned by the scheduler to run a job), the command-line client, and the web-ui. For information about working on the scheduler or the webUI, see the file &ldquo;developing-aurora-scheduler.md&rdquo; in this directory.</p>
+
+<p>If you want to work on the command-line client, this is the place for you!</p>
+
+<p>The client is written in Python, and unlike the server side of things, we build the client using the Pants build tool, instead of Gradle. Pants is a tool that was built by twitter for handling builds of large collaborative systems. You can see a detailed explanation of
+pants <a href="http://pantsbuild.github.io/python-readme.html">here</a>.</p>
+
+<p>To build the client executable, run the following in a command-shell:</p>
+<pre class="highlight text">$ ./pants src/main/python/apache/aurora/client/cli:aurora2
+</pre>
+<p>This will produce a python executable <em>pex</em> file in <code>dist/aurora2.pex</code>. Pex files
+are fully self-contained executables: just copy the pex file into your path, and you&rsquo;ll be able to run it. For example, for a typical installation:</p>
+<pre class="highlight text">$ cp dist/aurora2.pex /usr/local/bin/aurora
+</pre>
+<p>To run all of the client tests:</p>
+<pre class="highlight text">$ ./pasts src/test/python/apache/aurora/client/:all
+</pre>
+<h1 id="client-versions">Client Versions</h1>
+
+<p>There are currently two versions of the aurora client, imaginatively known as v1 and v2. All new development is done entirely in v2, but we continue to support and fix bugs in v1, until we get to the point where v2 is feature-complete and tested, and aurora users have had some time at adapt and switch their processes to use v2.</p>
+
+<p>Both versions are built on the same underlying API code.</p>
+
+<p>Client v1 was implemented using twitter.common.app. The command-line processing code for v1 can be found in <code>src/main/python/apache/aurora/client/commands</code> and
+<code>src/main/python/apache/aurora/client/bin</code>.</p>
+
+<p>Client v2 was implemented using its own noun/verb framework. The client v2 code can be found in <code>src/main/python/apache/aurora/client/cli</code>, and the noun/verb framework can be
+found in the <code>__init__.py</code> file in that directory.</p>
+
+<h1 id="building-and-testing-the-client">Building and Testing the Client</h1>
+
+<p>Building and testing the client code are both done using Pants. The relevant targets to know about are:</p>
+
+<ul>
+<li>Build a client v2 executable: <code>./pants src/main/python/apache/aurora/client/cli:aurora2</code></li>
+<li>Test client v2 code: <code>./pants ./pants src/test/python/apache/aurora/client/cli:all</code></li>
+<li>Build a client v1 executable: <code>./pants src/main/python/apache/aurora/client/bin:aurora_client</code></li>
+<li>Test client v1 code: <code>./pants src/main/python/apache/aurora/client/commands:all</code></li>
+<li>Test all client code: <code>./pants src/main/python/apache/aurora/client:all</code></li>
+</ul>
+
+<h1 id="overview-of-the-client-architecture">Overview of the Client Architecture</h1>
+
+<p>The client is built on a stacked architecture:</p>
+
+<ol>
+<li><p>At the lowest level, we have a thrift RPC API interface
+to the aurora scheduler. The interface is declared in thrift, in the file
+<code>src/main/thrift/org/apache/aurora/gen/api.thrift</code>.</p>
+
+<ol>
+<li>On top of the primitive API, we have a client API. The client API
+takes the primitive operations provided by the scheduler, and uses them
+to implement client-side behaviors. For example, when you update a job,
+on the scheduler, that&rsquo;s done by a sequence of operations.  The sequence is implemented
+by the client API <code>update</code> method, which does the following using the thrift API:
+
+<ul>
+<li>fetching the state of task instances in the mesos cluster, and figuring out which need
+to be updated;</li>
+<li>For each task to be updated:
+
+<ul>
+<li>killing the old version;</li>
+<li>starting the new version;</li>
+<li>monitoring the new version to ensure that the update succeeded.</li>
+</ul></li>
+</ul></li>
+<li>On top of the API, we have the command-line client itself. The core client, at this level,
+consists of the interface to the command-line which the user will use to interact with aurora.
+The client v2 code is found in <code>src/python/apache/aurora/client/cli</code>. In the <code>cli</code> directory,
+the rough structure is as follows:
+
+<ul>
+<li><code>__init__.py</code> contains the noun/verb command-line processing framework used by client v2.</li>
+<li><code>jobs.py</code> contains the implementation of the core <code>job</code> noun, and all of its operations.</li>
+<li><code>bridge.py</code> contains the implementation of a component that allows us to ship a
+combined client that runs both v1 and v2 client commands during the transition period.</li>
+<li><code>client.py</code> contains the code that binds the client v2 nouns and verbs into an executable.</li>
+</ul></li>
+</ol></li>
+</ol>
+
+<h1 id="running/debugging-the-client">Running/Debugging the Client</h1>
+
+<p>For manually testing client changes against a cluster, we use vagrant. To start a virtual cluster,
+you need to install a working vagrant environment, and then run &ldquo;vagrant up&rdquo; for the root of
+the aurora workspace. This will create a vagrant host named &ldquo;devcluster&rdquo;, with a mesos master,
+a set of mesos slaves, and an aurora scheduler.</p>
+
+<p>To use the devcluster, you need to bring it up by running <code>vagrant up</code>, and then connect to the vagrant host using <code>vagrant ssh</code>. This will open a bash session on the virtual machine hosting the devcluster. In the home directory, there are two key paths to know about:</p>
+
+<ul>
+<li><code>~/aurora</code>: this is a copy of the git workspace in which you launched the vagrant cluster.
+ To test client changes, you&rsquo;ll use this copy.</li>
+<li><code>/vagrant</code>: this is a mounted filesystem that&rsquo;s a direct image of your git workspace.
+ This isn&rsquo;t a copy - it is your git workspace. Editing files on your host machine will
+ be immediately visible here, because they are the same files.</li>
+</ul>
+
+<p>Whenever the scheduler is modified, to update your vagrant environment to use the new scheduler,
+you&rsquo;ll need to re-initialize your vagrant images. To do this, you need to run two commands:</p>
+
+<ul>
+<li><code>vagrant destroy</code>: this will delete the old devcluster image.</li>
+<li><code>vagrant up</code>: this creates a fresh devcluster image based on the current state of your workspace.</li>
+</ul>
+
+<p>You should try to minimize rebuilding vagrant images; it&rsquo;s not horribly slow, but it does take a while.</p>
+
+<p>To test client changes:</p>
+
+<ul>
+<li>Make a change in your local workspace, and commit it.</li>
+<li><code>vagrant ssh</code> into the devcluster.</li>
+<li><code>cd aurora</code></li>
+<li>Pull your changes into the vagrant copy: <code>git pull /vagrant *branchname*</code>.</li>
+<li>Build the modified client using pants.</li>
+<li>Run your command using <code>aurora2</code>. (You don&rsquo;t need to do any install; the aurora2 command
+ is a symbolic link to the executable generated by pants.)</li>
+</ul>
+
+	  </div>
+      <div class="container">
+    <hr>
+    <footer class="footer">
+        <div class="row-fluid">
+            <div class="span2 text-left">
+                <h3>Links</h3>
+                <ul class="unstyled">
+                    <li><a href="/downloads/">Downloads</a></li>
+                    <li><a href="/developers/">Developers</a></li>                    
+                </ul>
+            </div>
+            <div class="span3 text-left">
+                <h3>Community</h3>
+                <ul class="unstyled">
+                    <li><a href="/community/">Mailing Lists</a></li>
+                    <li><a href="http://issues.apache.org/jira/browse/aurora">Issue Tracking</a></li>
+                    <li><a href="/docs/howtocontribute/">How To Contribute</a></li>
+                </ul>
+            </div>
+            <div class="span7 text-left">
+            	<h3>Apache Software Foundation</h3>
+
+							<div class="span8">
+                Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. Apache, Apache Thrift, and the Apache feather logo are trademarks of The Apache Software Foundation. Currently part of the <a href="http://incubator.apache.org">Apache Incubator</a>.
+							</div>
+							<div class=" pull-right">
+								<a href="http://incubator.apache.org" class="logo"><img src="/assets/img/apache_incubator_logo.png" alt="Apache Incubator" class="pull-right"/></a>
+							</div>
+            </div>
+
+        </div>
+
+    </footer>
+</div>
+
+	</body>
+</html>
+

Modified: incubator/aurora/site/publish/documentation/latest/developing-aurora-scheduler/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/developing-aurora-scheduler/index.html?rev=1623604&r1=1623603&r2=1623604&view=diff
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/developing-aurora-scheduler/index.html (original)
+++ incubator/aurora/site/publish/documentation/latest/developing-aurora-scheduler/index.html Tue Sep  9 00:47:19 2014
@@ -84,6 +84,13 @@ run it but subsequent runs will be much 
 tests use</p>
 <pre class="highlight text">./gradlew clean build
 </pre>
+<h2 id="running-the-build-with-code-quality-checks">Running the build with code quality checks</h2>
+
+<p>To speed up development iteration, the plain gradle commands will not run static analysis tools.
+However, you should run these before posting a review diff, and <strong>always</strong> run this before pushing a
+commit to origin/master.</p>
+<pre class="highlight text">./gradlew build -Pq
+</pre>
 <h2 id="creating-a-bundle-for-deployment">Creating a bundle for deployment</h2>
 
 <p>Gradle can create a zip file containing Aurora, all of its dependencies, and a launch script with</p>

Added: incubator/aurora/site/publish/documentation/latest/sla/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/sla/index.html?rev=1623604&view=auto
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/sla/index.html (added)
+++ incubator/aurora/site/publish/documentation/latest/sla/index.html Tue Sep  9 00:47:19 2014
@@ -0,0 +1,302 @@
+<html>
+    <head>
+        <meta charset="utf-8">
+        <title>Apache Aurora</title>
+		    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+		    <meta name="description" content="">
+		    <meta name="author" content="">
+
+		    <link href="/assets/css/bootstrap.css" rel="stylesheet">
+		    <link href="/assets/css/bootstrap-responsive.min.css" rel="stylesheet">
+		    <link href="/assets/css/main.css" rel="stylesheet">
+				
+		    <!-- JS -->
+		    <script type="text/javascript" src="/assets/js/jquery-1.10.1.min.js"></script>
+		    <script type="text/javascript" src="/assets/js/bootstrap-dropdown.js"></script>
+		
+				<!-- Analytics -->
+				<script type="text/javascript">
+					  var _gaq = _gaq || [];
+					  _gaq.push(['_setAccount', 'UA-45879646-1']);
+					  _gaq.push(['_setDomainName', 'apache.org']);
+					  _gaq.push(['_trackPageview']);
+
+					  (function() {
+					    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+					    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+					    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+					  })();
+				</script>
+	</head>
+    <body>	
+      <div class="navbar navbar-static-top">
+  <div class="navbar-inner">
+    <div class="container">
+	    <a href="/" class="logo"><img src="/assets/img/aurora_logo.png" alt="Apache Aurora logo" /></a>
+      <ul class="nav">
+				<li><a href="/documentation/latest/">Documentation</a></li>
+        <li><a href="/downloads/">Download</a></li>
+        <li><a href="/community">Community</a></li>
+      </ul>
+    </div>
+  </div>
+</div>
+
+<div class="container">
+<!-- magical breadcrumbs -->
+<ul class="breadcrumb">
+  <li>
+    <div class="dropdown">
+      <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a>
+      <ul class="dropdown-menu" role="menu">
+        <li><a href="http://www.apache.org">Apache Homepage</a></li>
+        <li><a href="http://www.apache.org/licenses/">Apache License</a></li>
+        <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>  
+        <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+        <li><a href="http://www.apache.org/security/">Security</a></li>
+      </ul>
+    </div>
+  </li>
+  <li><span class="divider">&bull;</span></li>
+  <li><a href="http://incubator.apache.org">Apache Incubator</a></li>
+  <li><span class="divider">&bull;</span></li>
+  <li><a href="http://aurora.incubator.apache.org">Apache Aurora</a></li>
+</ul>
+<!-- /breadcrumb -->
+	
+      <div class="container">
+        <h2 id="aurora-sla-measurement">Aurora SLA Measurement</h2>
+
+<ul>
+<li><a href="#overview">Overview</a></li>
+<li><a href="#metric-details">Metric Details</a>
+
+<ul>
+<li><a href="#platform-uptime">Platform Uptime</a></li>
+<li><a href="#job-uptime">Job Uptime</a></li>
+<li><a href="#median-time-to-assigned-(mtta)">Median Time To Assigned (MTTA)</a></li>
+<li><a href="#median-time-to-running-(mttr)">Median Time To Running (MTTR)</a></li>
+</ul></li>
+<li><a href="#limitations">Limitations</a></li>
+</ul>
+
+<h2 id="overview">Overview</h2>
+
+<p>The primary goal of the feature is collection and monitoring of Aurora job SLA (Service Level
+Agreements) metrics that defining a contractual relationship between the Aurora/Mesos platform
+and hosted services.</p>
+
+<p>The Aurora SLA feature currently supports stat collection only for service (non-cron)
+production jobs (<code>&quot;production = True&quot;</code> in your <code>.aurora</code> config).</p>
+
+<p>Counters that track SLA measurements are computed periodically within the scheduler.
+The individual instance metrics are refreshed every minute (configurable via
+<code>sla_stat_refresh_interval</code>). The instance counters are subsequently aggregated by
+relevant grouping types before exporting to scheduler <code>/vars</code> endpoint (when using <code>vagrant</code>
+that would be <code>http://192.168.33.7:8081/vars</code>)</p>
+
+<h2 id="metric-details">Metric Details</h2>
+
+<h3 id="platform-uptime">Platform Uptime</h3>
+
+<p><em>Aggregate amount of time a job spends in a non-runnable state due to platform unavailability
+or scheduling delays. This metric tracks Aurora/Mesos uptime performance and reflects on any
+system-caused downtime events (tasks LOST or DRAINED). Any user-initiated task kills/restarts
+will not degrade this metric.</em></p>
+
+<p><strong>Collection scope:</strong></p>
+
+<ul>
+<li>Per job - <code>sla_&lt;job_key&gt;_platform_uptime_percent</code></li>
+<li>Per cluster - <code>sla_cluster_platform_uptime_percent</code></li>
+</ul>
+
+<p><strong>Units:</strong> percent</p>
+
+<p>A fault in the task environment may cause the Aurora/Mesos to have different views on the task state
+or lose track of the task existence. In such cases, the service task is marked as LOST and
+rescheduled by Aurora. For example, this may happen when the task stays in ASSIGNED or STARTING
+for too long or the Mesos slave becomes unhealthy (or disappears completely). The time between
+task entering LOST and its replacement reaching RUNNING state is counted towards platform downtime.</p>
+
+<p>Another example of a platform downtime event is the administrator-requested task rescheduling. This
+happens during planned Mesos slave maintenance when all slave tasks are marked as DRAINED and
+rescheduled elsewhere.</p>
+
+<p>To accurately calculate Platform Uptime, we must separate platform incurred downtime from user
+actions that put a service instance in a non-operational state. It is simpler to isolate
+user-incurred downtime and treat all other downtime as platform incurred.</p>
+
+<p>Currently, a user can cause a healthy service (task) downtime in only two ways: via <code>killTasks</code>
+or <code>restartShards</code> RPCs. For both, their affected tasks leave an audit state transition trail
+relevant to uptime calculations. By applying a special &ldquo;SLA meaning&rdquo; to exposed task state
+transition records, we can build a deterministic downtime trace for every given service instance.</p>
+
+<p>A task going through a state transition carries one of three possible SLA meanings
+(see <a href="../src/main/java/org/apache/aurora/scheduler/sla/SlaAlgorithm.java">SlaAlgorithm.java</a> for
+sla-to-task-state mapping):</p>
+
+<ul>
+<li><p>Task is UP: starts a period where the task is considered to be up and running from the Aurora
+platform standpoint.</p></li>
+<li><p>Task is DOWN: starts a period where the task cannot reach the UP state for some
+non-user-related reason. Counts towards instance downtime.</p></li>
+<li><p>Task is REMOVED from SLA: starts a period where the task is not expected to be UP due to
+user initiated action or failure. We ignore this period for the uptime calculation purposes.</p></li>
+</ul>
+
+<p>This metric is recalculated over the last sampling period (last minute) to account for
+any UP/DOWN/REMOVED events. It ignores any UP/DOWN events not immediately adjacent to the
+sampling interval as well as adjacent REMOVED events.</p>
+
+<h3 id="job-uptime">Job Uptime</h3>
+
+<p><em>Percentage of the job instances considered to be in RUNNING state for the specified duration
+relative to request time. This is a purely application side metric that is considering aggregate
+uptime of all RUNNING instances. Any user- or platform initiated restarts directly affect
+this metric.</em></p>
+
+<p><strong>Collection scope:</strong> We currently expose job uptime values at 5 pre-defined
+percentiles (50th,75th,90th,95th and 99th):</p>
+
+<ul>
+<li><code>sla_&lt;job_key&gt;_job_uptime_50_00_sec</code></li>
+<li><code>sla_&lt;job_key&gt;_job_uptime_75_00_sec</code></li>
+<li><code>sla_&lt;job_key&gt;_job_uptime_90_00_sec</code></li>
+<li><code>sla_&lt;job_key&gt;_job_uptime_95_00_sec</code></li>
+<li><code>sla_&lt;job_key&gt;_job_uptime_99_00_sec</code></li>
+</ul>
+
+<p><strong>Units:</strong> seconds
+You can also get customized real-time stats from aurora client. See <code>aurora sla -h</code> for
+more details.</p>
+
+<h3 id="median-time-to-assigned-(mtta)">Median Time To Assigned (MTTA)</h3>
+
+<p><em>Median time a job spends waiting for its tasks to be assigned to a host. This is a combined
+metric that helps track the dependency of scheduling performance on the requested resources
+(user scope) as well as the internal scheduler bin-packing algorithm efficiency (platform scope).</em></p>
+
+<p><strong>Collection scope:</strong></p>
+
+<ul>
+<li>Per job - <code>sla_&lt;job_key&gt;_mtta_ms</code></li>
+<li>Per cluster - <code>sla_cluster_mtta_ms</code></li>
+<li>Per instance size (small, medium, large, x-large, xx-large). Size are defined in:
+<a href="../src/main/java/org/apache/aurora/scheduler/base/ResourceAggregates.java">ResourceAggregates.java</a>
+
+<ul>
+<li>By CPU:</li>
+<li><code>sla_cpu_small_mtta_ms</code></li>
+<li><code>sla_cpu_medium_mtta_ms</code></li>
+<li><code>sla_cpu_large_mtta_ms</code></li>
+<li><code>sla_cpu_xlarge_mtta_ms</code></li>
+<li><code>sla_cpu_xxlarge_mtta_ms</code></li>
+<li>By RAM:</li>
+<li><code>sla_ram_small_mtta_ms</code></li>
+<li><code>sla_ram_medium_mtta_ms</code></li>
+<li><code>sla_ram_large_mtta_ms</code></li>
+<li><code>sla_ram_xlarge_mtta_ms</code></li>
+<li><code>sla_ram_xxlarge_mtta_ms</code></li>
+<li>By DISK:</li>
+<li><code>sla_disk_small_mtta_ms</code></li>
+<li><code>sla_disk_medium_mtta_ms</code></li>
+<li><code>sla_disk_large_mtta_ms</code></li>
+<li><code>sla_disk_xlarge_mtta_ms</code></li>
+<li><code>sla_disk_xxlarge_mtta_ms</code></li>
+</ul></li>
+</ul>
+
+<p><strong>Units:</strong> milliseconds</p>
+
+<p>MTTA only considers instances that have already reached ASSIGNED state and ignores those
+that are still PENDING. This ensures straggler instances (e.g. with unreasonable resource
+constraints) do not affect metric curves.</p>
+
+<h3 id="median-time-to-running-(mttr)">Median Time To Running (MTTR)</h3>
+
+<p><em>Median time a job waits for its tasks to reach RUNNING state. This is a comprehensive metric
+reflecting on the overall time it takes for the Aurora/Mesos to start executing user content.</em></p>
+
+<p><strong>Collection scope:</strong></p>
+
+<ul>
+<li>Per job - <code>sla_&lt;job_key&gt;_mttr_ms</code></li>
+<li>Per cluster - <code>sla_cluster_mttr_ms</code></li>
+<li>Per instance size (small, medium, large, x-large, xx-large). Size are defined in:
+<a href="../src/main/java/org/apache/aurora/scheduler/base/ResourceAggregates.java">ResourceAggregates.java</a>
+
+<ul>
+<li>By CPU:</li>
+<li><code>sla_cpu_small_mttr_ms</code></li>
+<li><code>sla_cpu_medium_mttr_ms</code></li>
+<li><code>sla_cpu_large_mttr_ms</code></li>
+<li><code>sla_cpu_xlarge_mttr_ms</code></li>
+<li><code>sla_cpu_xxlarge_mttr_ms</code></li>
+<li>By RAM:</li>
+<li><code>sla_ram_small_mttr_ms</code></li>
+<li><code>sla_ram_medium_mttr_ms</code></li>
+<li><code>sla_ram_large_mttr_ms</code></li>
+<li><code>sla_ram_xlarge_mttr_ms</code></li>
+<li><code>sla_ram_xxlarge_mttr_ms</code></li>
+<li>By DISK:</li>
+<li><code>sla_disk_small_mttr_ms</code></li>
+<li><code>sla_disk_medium_mttr_ms</code></li>
+<li><code>sla_disk_large_mttr_ms</code></li>
+<li><code>sla_disk_xlarge_mttr_ms</code></li>
+<li><code>sla_disk_xxlarge_mttr_ms</code></li>
+</ul></li>
+</ul>
+
+<p><strong>Units:</strong> milliseconds</p>
+
+<p>MTTR only considers instances in RUNNING state. This ensures straggler instances (e.g. with
+unreasonable resource constraints) do not affect metric curves.</p>
+
+<h2 id="limitations">Limitations</h2>
+
+<ul>
+<li><p>The availability of Aurora SLA metrics is bound by the scheduler availability.</p></li>
+<li><p>All metrics are calculated at a pre-defined interval (currently set at 1 minute).
+Scheduler restarts may result in missed collections.</p></li>
+</ul>
+
+	  </div>
+      <div class="container">
+    <hr>
+    <footer class="footer">
+        <div class="row-fluid">
+            <div class="span2 text-left">
+                <h3>Links</h3>
+                <ul class="unstyled">
+                    <li><a href="/downloads/">Downloads</a></li>
+                    <li><a href="/developers/">Developers</a></li>                    
+                </ul>
+            </div>
+            <div class="span3 text-left">
+                <h3>Community</h3>
+                <ul class="unstyled">
+                    <li><a href="/community/">Mailing Lists</a></li>
+                    <li><a href="http://issues.apache.org/jira/browse/aurora">Issue Tracking</a></li>
+                    <li><a href="/docs/howtocontribute/">How To Contribute</a></li>
+                </ul>
+            </div>
+            <div class="span7 text-left">
+            	<h3>Apache Software Foundation</h3>
+
+							<div class="span8">
+                Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. Apache, Apache Thrift, and the Apache feather logo are trademarks of The Apache Software Foundation. Currently part of the <a href="http://incubator.apache.org">Apache Incubator</a>.
+							</div>
+							<div class=" pull-right">
+								<a href="http://incubator.apache.org" class="logo"><img src="/assets/img/apache_incubator_logo.png" alt="Apache Incubator" class="pull-right"/></a>
+							</div>
+            </div>
+
+        </div>
+
+    </footer>
+</div>
+
+	</body>
+</html>
+

Modified: incubator/aurora/site/publish/documentation/latest/user-guide/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/user-guide/index.html?rev=1623604&r1=1623603&r2=1623604&view=diff
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/user-guide/index.html (original)
+++ incubator/aurora/site/publish/documentation/latest/user-guide/index.html Tue Sep  9 00:47:19 2014
@@ -68,20 +68,26 @@
         <h2 id="aurora-user-guide">Aurora User Guide</h2>
 
 <ul>
-<li><a href="#overview">Overview</a></li>
-<li><a href="#job-lifecycle">Job Lifecycle</a>
+<li><a href="#user-content-overview">Overview</a></li>
+<li><a href="#user-content-job-lifecycle">Job Lifecycle</a>
 
 <ul>
-<li><a href="#life-of-a-task">Life Of A Task</a></li>
-<li><a href="#pending-to-running-states">PENDING to RUNNING states</a></li>
-<li><a href="#task-updates">Task Updates</a></li>
-<li><a href="#giving-priority-to-production-tasks-preempting">Giving Priority to Production Tasks: PREEMPTING</a></li>
-<li><a href="#natural-termination-finished-failed">Natural Termination: FINISHED, FAILED</a></li>
-<li><a href="#forceful-termination-killing-restarting">Forceful Termination: KILLING, RESTARTING</a></li>
+<li><a href="#user-content-life-of-a-task">Life Of A Task</a></li>
+<li><a href="#user-content-pending-to-running-states">PENDING to RUNNING states</a></li>
+<li><a href="#user-content-task-updates">Task Updates</a></li>
+<li><a href="#user-content-http-health-checking-and-graceful-shutdown">HTTP Health Checking and Graceful Shutdown</a>
+
+<ul>
+<li><a href="#user-content-tearing-a-task-down">Tearing a task down</a></li>
+</ul></li>
+<li><a href="#user-content-giving-priority-to-production-tasks-preempting">Giving Priority to Production Tasks: PREEMPTING</a></li>
+<li><a href="#user-content-natural-termination-finished-failed">Natural Termination: FINISHED, FAILED</a></li>
+<li><a href="#user-content-forceful-termination-killing-restarting">Forceful Termination: KILLING, RESTARTING</a></li>
 </ul></li>
-<li><a href="#configuration">Configuration</a></li>
-<li><a href="#creating-jobs">Creating Jobs</a></li>
-<li><a href="#interacting-with-jobs">Interacting With Jobs</a></li>
+<li><a href="#user-content-service-discovery">Service Discovery</a></li>
+<li><a href="#user-content-configuration">Configuration</a></li>
+<li><a href="#user-content-creating-jobs">Creating Jobs</a></li>
+<li><a href="#user-content-interacting-with-jobs">Interacting With Jobs</a></li>
 </ul>
 
 <h2 id="overview">Overview</h2>
@@ -182,9 +188,6 @@ checkpointing mechanisms directly into y
 <code>Task</code>&rsquo;s life cycle.</li>
 </ol>
 
-<p><strong>Note</strong>: It is not currently possible to create an Aurora job from
-within an Aurora job.</p>
-
 <h3 id="life-of-a-task">Life Of A Task</h3>
 
 <p><img alt="Life of a task" src="../images/lifeofatask.png" /></p>
@@ -263,6 +266,64 @@ with old instance configs and batch upda
 from the point where the update failed. E.g.; (0,1,2) (3,4,5) (6,7,
 8-FAIL) results in a rollback in order (8,7,6) (5,4,3) (2,1,0).</p>
 
+<h3 id="http-health-checking-and-graceful-shutdown">HTTP Health Checking and Graceful Shutdown</h3>
+
+<p>The Executor implements a protocol for rudimentary control of a task via HTTP.  Tasks subscribe for
+this protocol by declaring a port named <code>health</code>.  Take for example this configuration snippet:</p>
+<pre class="highlight text">nginx = Process(
+  name = &#39;nginx&#39;,
+  cmdline = &#39;./run_nginx.sh -port {{thermos.ports[http]}}&#39;)
+</pre>
+<p>When this Process is included in a job, the job will be allocated a port, and the command line
+will be replaced with something like:</p>
+<pre class="highlight text">./run_nginx.sh -port 42816
+</pre>
+<p>Where 42816 happens to be the allocated. port.  Typically, the Executor monitors Processes within
+a task only by liveness of the forked process.  However, when a <code>health</code> port was allocated, it will
+also send periodic HTTP health checks.  A task requesting a <code>health</code> port must handle the following
+requests:</p>
+
+<table><thead>
+<tr>
+<th>HTTP request</th>
+<th>Description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>GET /health</code></td>
+<td>Inquires whether the task is healthy.</td>
+</tr>
+<tr>
+<td><code>POST /quitquitquit</code></td>
+<td>Task should initiate graceful shutdown.</td>
+</tr>
+<tr>
+<td><code>POST /abortabortabort</code></td>
+<td>Final warning task is being killed.</td>
+</tr>
+</tbody></table>
+
+<p>Please see the
+<a href="configuration-reference.md#user-content-healthcheckconfig-objects">configuration reference</a> for
+configuration options for this feature.</p>
+
+<h4 id="tearing-a-task-down">Tearing a task down</h4>
+
+<p>The Executor follows an escalation sequence when killing a running task:</p>
+
+<ol>
+<li>If <code>health</code> port is not present, skip to (5)</li>
+<li>POST /quitquitquit</li>
+<li>wait 5 seconds</li>
+<li>POST /abortabortabort</li>
+<li>Send SIGTERM (<code>kill</code>)</li>
+<li>Send SIGKILL (<code>kill -9</code>)</li>
+</ol>
+
+<p>If the Executor notices that all Processes in a Task have aborted during this sequence, it will
+not proceed with subsequent steps.  Note that graceful shutdown is best-effort, and due to the many
+inevitable realities of distributed systems, it may not be performed.</p>
+
 <h3 id="giving-priority-to-production-tasks:-preempting">Giving Priority to Production Tasks: PREEMPTING</h3>
 
 <p>Sometimes a Task needs to be interrupted, such as when a non-production
@@ -311,12 +372,28 @@ it.</p>
 
 <p>You define and configure your Jobs (and their Tasks and Processes) in
 Aurora configuration files. Their filenames end with the <code>.aurora</code>
-suffix, and you write them in Python making use of the Pystashio
+suffix, and you write them in Python making use of the Pystachio
 templating language, along
 with specific Aurora, Mesos, and Thermos commands and methods. See the
 <a href="/documentation/latest/configuration-reference/">Configuration Guide and Reference</a> and
 <a href="/documentation/latest/configuration-tutorial/">Configuration Tutorial</a>.</p>
 
+<h2 id="service-discovery">Service Discovery</h2>
+
+<p>It is possible for the Aurora executor to announce tasks into ServerSets for
+the purpose of service discovery.  ServerSets use the Zookeeper <a href="http://zookeeper.apache.org/doc/trunk/recipes.html#sc_outOfTheBox">group membership pattern</a>
+of which there are several reference implementations:</p>
+
+<ul>
+<li><a href="https://github.com/apache/mesos/blob/master/src/zookeeper/group.cpp">C++</a></li>
+<li><a href="https://github.com/twitter/commons/blob/master/src/java/com/twitter/common/zookeeper/ServerSetImpl.java#L221">Java</a></li>
+<li><a href="https://github.com/twitter/commons/blob/master/src/python/twitter/common/zookeeper/serverset/serverset.py#L51">Python</a></li>
+</ul>
+
+<p>These can also be used natively in Finagle using the <a href="https://github.com/twitter/finagle/blob/master/finagle-serversets/src/main/scala/com/twitter/finagle/zookeeper/ZookeeperServerSetCluster.scala">ZookeeperServerSetCluster</a>.</p>
+
+<p>For more information about how to configure announcing, see the <a href="/documentation/latest/configuration-reference/">Configuration Reference</a>.</p>
+
 <h2 id="creating-jobs">Creating Jobs</h2>
 
 <p>You create and manipulate Aurora Jobs with the Aurora client, which starts all its



Mime
View raw message