aurora-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jfarr...@apache.org
Subject [12/18] incubator-aurora-website git commit: Initial move of website over to git
Date Wed, 17 Dec 2014 18:30:37 GMT
http://git-wip-us.apache.org/repos/asf/incubator-aurora-website/blob/c43a3a2d/publish/documentation/latest/configuration-tutorial/index.html
----------------------------------------------------------------------
diff --git a/publish/documentation/latest/configuration-tutorial/index.html b/publish/documentation/latest/configuration-tutorial/index.html
new file mode 100644
index 0000000..3468077
--- /dev/null
+++ b/publish/documentation/latest/configuration-tutorial/index.html
@@ -0,0 +1,1218 @@
+<!DOCTYPE html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+	<title>Apache Aurora</title>
+    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+    <link href="/assets/css/main.css" rel="stylesheet">
+	<!-- Analytics -->
+	<script type="text/javascript">
+		  var _gaq = _gaq || [];
+		  _gaq.push(['_setAccount', 'UA-45879646-1']);
+		  _gaq.push(['_setDomainName', 'apache.org']);
+		  _gaq.push(['_trackPageview']);
+
+		  (function() {
+		    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+		    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+		    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+		  })();
+	</script>
+  </head>
+  <body>
+	  
+        <div class="container-fluid section-header">
+  <div class="container">
+    <div class="nav nav-bar">
+    <a href="/"><img src="/assets/img/aurora_logo_white_bkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+	<ul class="nav navbar-nav navbar-right">
+      <li><a href="/documentation/latest/">Documentation</a></li>
+      <li><a href="/community/">Community</a></li>
+      <li><a href="/downloads/">Downloads</a></li>
+      <li><a href="/blog/">Blog</a></li>
+    </ul>
+    </div>
+  </div>
+</div>	
+  	  <div class="container-fluid">
+  	  	<div class="container content">
+          <h1 id="aurora-configuration-tutorial">Aurora Configuration Tutorial</h1>
+
+<p>How to write Aurora configuration files, including feature descriptions
+and best practices. When writing a configuration file, make use of
+<code>aurora inspect</code>. It takes the same job key and configuration file
+arguments as <code>aurora create</code> or <code>aurora update</code>. It first ensures the
+configuration parses, then outputs it in human-readable form.</p>
+
+<p>You should read this after going through the general <a href="/documentation/latest/tutorial/">Aurora Tutorial</a>.</p>
+
+<ul>
+<li><a href="#user-content-aurora-configuration-tutorial">Aurora Configuration Tutorial</a>
+
+<ul>
+<li><a href="#user-content-the-basics">The Basics</a>
+
+<ul>
+<li><a href="#user-content-use-bottom-to-top-object-ordering">Use Bottom-To-Top Object Ordering</a></li>
+</ul></li>
+<li><a href="#user-content-an-example-configuration-file">An Example Configuration File</a></li>
+<li><a href="#user-content-defining-process-objects">Defining Process Objects</a></li>
+<li><a href="#user-content-getting-your-code-into-the-sandbox">Getting Your Code Into The Sandbox</a></li>
+<li><a href="#user-content-defining-task-objects">Defining Task Objects</a>
+
+<ul>
+<li><a href="#user-content-sequentialtask-running-processes-in-parallel-or-sequentially">SequentialTask: Running Processes in Parallel or Sequentially</a></li>
+<li><a href="#user-content-simpletask">SimpleTask</a></li>
+<li><a href="#user-content-combining-tasks">Combining tasks</a></li>
+</ul></li>
+<li><a href="#user-content-defining-job-objects">Defining Job Objects</a></li>
+<li><a href="#user-content-the-jobs-list">The jobs List</a></li>
+<li><a href="#user-content-templating">Templating</a>
+
+<ul>
+<li><a href="#user-content-templating-1-binding-in-pystachio">Templating 1: Binding in Pystachio</a></li>
+<li><a href="#user-content-structurals-in-pystachio--aurora">Structurals in Pystachio / Aurora</a>
+
+<ul>
+<li><a href="#user-content-mustaches-within-structurals">Mustaches Within Structurals</a></li>
+</ul></li>
+<li><a href="#user-content-templating-2-structurals-are-factories">Templating 2: Structurals Are Factories</a>
+
+<ul>
+<li><a href="#user-content-a-second-way-of-templating">A Second Way of Templating</a></li>
+</ul></li>
+<li><a href="#user-content-advanced-binding">Advanced Binding</a>
+
+<ul>
+<li><a href="#user-content-bind-syntax">Bind Syntax</a></li>
+<li><a href="#user-content-binding-complex-objects">Binding Complex Objects</a>
+
+<ul>
+<li><a href="#user-content-lists"></a></li>
+<li><a href="#user-content-maps"></a></li>
+<li><a href="#user-content-structurals"></a></li>
+</ul></li>
+</ul></li>
+<li><a href="#user-content-structural-binding">Structural Binding</a></li>
+</ul></li>
+<li><a href="#user-content-configuration-file-writing-tips-and-best-practices">Configuration File Writing Tips And Best Practices</a>
+
+<ul>
+<li><a href="#user-content-use-as-few-aurora-files-as-possible">Use As Few .aurora Files As Possible</a></li>
+<li><a href="#user-content-avoid-boilerplate">Avoid Boilerplate</a></li>
+<li><a href="#user-content-thermos-uses-bash-but-thermos-is-not-bash">Thermos Uses bash, But Thermos Is Not bash</a>
+
+<ul>
+<li><a href="#user-content-bad">Bad</a></li>
+<li><a href="#user-content-good">Good</a></li>
+</ul></li>
+<li><a href="#user-content-rarely-use-functions-in-your-configurations">Rarely Use Functions In Your Configurations</a>
+
+<ul>
+<li><a href="#user-content-bad-1">Bad</a></li>
+<li><a href="#user-content-good-1">Good</a></li>
+</ul></li>
+</ul></li>
+</ul></li>
+</ul>
+
+<h2 id="the-basics">The Basics</h2>
+
+<p>To run a job on Aurora, you must specify a configuration file that tells
+Aurora what it needs to know to schedule the job, what Mesos needs to
+run the tasks the job is made up of, and what Thermos needs to run the
+processes that make up the tasks. This file must have
+a<code>.aurora</code> suffix.</p>
+
+<p>A configuration file defines a collection of objects, along with parameter
+values for their attributes. An Aurora configuration file contains the
+following three types of objects:</p>
+
+<ul>
+<li>Job</li>
+<li>Task</li>
+<li>Process</li>
+</ul>
+
+<p>A configuration also specifies a list of <code>Job</code> objects assigned
+to the variable <code>jobs</code>.</p>
+
+<ul>
+<li>jobs (list of defined Jobs to run)</li>
+</ul>
+
+<p>The <code>.aurora</code> file format is just Python. However, <code>Job</code>, <code>Task</code>,
+<code>Process</code>, and other classes are defined by a type-checked dictionary
+templating library called <em>Pystachio</em>, a powerful tool for
+configuration specification and reuse. Pystachio objects are tailored
+via {{}} surrounded templates.</p>
+
+<p>When writing your <code>.aurora</code> file, you may use any Pystachio datatypes, as
+well as any objects shown in the <a href="/documentation/latest/configuration-reference/"><em>Aurora+Thermos Configuration
+Reference</em></a>, without <code>import</code> statements - the
+Aurora config loader injects them automatically. Other than that, an <code>.aurora</code>
+file works like any other Python script.</p>
+
+<p><a href="/documentation/latest/configuration-reference/"><em>Aurora+Thermos Configuration Reference</em></a>
+has a full reference of all Aurora/Thermos defined Pystachio objects.</p>
+
+<h3 id="use-bottom-to-top-object-ordering">Use Bottom-To-Top Object Ordering</h3>
+
+<p>A well-structured configuration starts with structural templates (if
+any). Structural templates encapsulate in their attributes all the
+differences between Jobs in the configuration that are not directly
+manipulated at the <code>Job</code> level, but typically at the <code>Process</code> or <code>Task</code>
+level. For example, if certain processes are invoked with slightly
+different settings or input.</p>
+
+<p>After structural templates, define, in order, <code>Process</code>es, <code>Task</code>s, and
+<code>Job</code>s.</p>
+
+<p>Structural template names should be <em>UpperCamelCased</em> and their
+instantiations are typically <em>UPPER_SNAKE_CASED</em>. <code>Process</code>, <code>Task</code>,
+and <code>Job</code> names are typically <em>lower_snake_cased</em>. Indentation is typically 2
+spaces.</p>
+
+<h2 id="an-example-configuration-file">An Example Configuration File</h2>
+
+<p>The following is a typical configuration file. Don&rsquo;t worry if there are
+parts you don&rsquo;t understand yet, but you may want to refer back to this
+as you read about its individual parts. Note that names surrounded by
+curly braces {{}} are template variables, which the system replaces with
+bound values for the variables.</p>
+<pre class="highlight text"># --- templates here ---
+class Profile(Struct):
+  package_version = Default(String, &#39;live&#39;)
+  java_binary = Default(String, &#39;/usr/lib/jvm/java-1.7.0-openjdk/bin/java&#39;)
+  extra_jvm_options = Default(String, &#39;&#39;)
+  parent_environment = Default(String, &#39;prod&#39;)
+  parent_serverset = Default(String,
+                             &#39;/foocorp/service/bird/{{parent_environment}}/bird&#39;)
+
+# --- processes here ---
+main = Process(
+  name = &#39;application&#39;,
+  cmdline = &#39;{{profile.java_binary}} -server -Xmx1792m &#39;
+            &#39;{{profile.extra_jvm_options}} &#39;
+            &#39;-jar application.jar &#39;
+            &#39;-upstreamService {{profile.parent_serverset}}&#39;
+)
+
+# --- tasks ---
+base_task = SequentialTask(
+  name = &#39;application&#39;,
+  processes = [
+    Process(
+      name = &#39;fetch&#39;,
+      cmdline = &#39;curl -O
+              https://packages.foocorp.com/{{profile.package_version}}/application.jar&#39;),
+  ]
+)
+
+    # not always necessary but often useful to have separate task
+    # resource classes
+    staging_task = base_task(resources =
+                     Resources(cpu = 1.0,
+                               ram = 2048*MB,
+                               disk = 1*GB))
+production_task = base_task(resources =
+                        Resources(cpu = 4.0,
+                                  ram = 2560*MB,
+                                  disk = 10*GB))
+
+# --- job template ---
+job_template = Job(
+  name = &#39;application&#39;,
+  role = &#39;myteam&#39;,
+  contact = &#39;myteam-team@foocorp.com&#39;,
+  instances = 20,
+  service = True,
+  task = production_task
+)
+
+# -- profile instantiations (if any) ---
+PRODUCTION = Profile()
+STAGING = Profile(
+  extra_jvm_options = &#39;-Xloggc:gc.log&#39;,
+  parent_environment = &#39;staging&#39;
+)
+
+# -- job instantiations --
+jobs = [
+      job_template(cluster = &#39;cluster1&#39;, environment = &#39;prod&#39;)
+               .bind(profile = PRODUCTION),
+
+      job_template(cluster = &#39;cluster2&#39;, environment = &#39;prod&#39;)
+                .bind(profile = PRODUCTION),
+
+      job_template(cluster = &#39;cluster1&#39;,
+                    environment = &#39;staging&#39;,
+        service = False,
+        task = staging_task,
+        instances = 2)
+        .bind(profile = STAGING),
+]
+</pre>
+<h2 id="defining-process-objects">Defining Process Objects</h2>
+
+<p>Processes are handled by the Thermos system. A process is a single
+executable step run as a part of an Aurora task, which consists of a
+bash-executable statement.</p>
+
+<p>The key (and required) <code>Process</code> attributes are:</p>
+
+<ul>
+<li>  <code>name</code>: Any string which is a valid Unix filename (no slashes,
+NULLs, or leading periods). The <code>name</code> value must be unique relative
+to other Processes in a <code>Task</code>.</li>
+<li>  <code>cmdline</code>: A command line run in a bash subshell, so you can use
+bash scripts. Nothing is supplied for command-line arguments,
+so <code>$*</code> is unspecified.</li>
+</ul>
+
+<p>Many tiny processes make managing configurations more difficult. For
+example, the following is a bad way to define processes.</p>
+<pre class="highlight text">copy = Process(
+  name = &#39;copy&#39;,
+  cmdline = &#39;curl -O https://packages.foocorp.com/app.zip&#39;
+)
+unpack = Process(
+  name = &#39;unpack&#39;,
+  cmdline = &#39;unzip app.zip&#39;
+)
+remove = Process(
+  name = &#39;remove&#39;,
+  cmdline = &#39;rm -f app.zip&#39;
+)
+run = Process(
+  name = &#39;app&#39;,
+  cmdline = &#39;java -jar app.jar&#39;
+)
+run_task = Task(
+  processes = [copy, unpack, remove, run],
+  constraints = order(copy, unpack, remove, run)
+)
+</pre>
+<p>Since <code>cmdline</code> runs in a bash subshell, you can chain commands
+with <code>&amp;&amp;</code> or <code>||</code>.</p>
+
+<p>When defining a <code>Task</code> that is just a list of Processes run in a
+particular order, use <code>SequentialTask</code>, as described in the <a href="#Task"><em>Defining</em>
+<code>Task</code> <em>Objects</em></a> section. The following simplifies and combines the
+above multiple <code>Process</code> definitions into just two.</p>
+<pre class="highlight text">stage = Process(
+  name = &#39;stage&#39;,
+  cmdline = &#39;curl -O https://packages.foocorp.com/app.zip &amp;&amp; &#39;
+            &#39;unzip app.zip &amp;&amp; rm -f app.zip&#39;)
+
+run = Process(name = &#39;app&#39;, cmdline = &#39;java -jar app.jar&#39;)
+
+run_task = SequentialTask(processes = [stage, run])
+</pre>
+<p><code>Process</code> also has five optional attributes, each with a default value
+if one isn&rsquo;t specified in the configuration:</p>
+
+<ul>
+<li><p><code>max_failures</code>: Defaulting to <code>1</code>, the maximum number of failures
+(non-zero exit statuses) before this <code>Process</code> is marked permanently
+failed and not retried. If a <code>Process</code> permanently fails, Thermos
+checks the <code>Process</code> object&rsquo;s containing <code>Task</code> for the task&rsquo;s
+failure limit (usually 1) to determine whether or not the <code>Task</code>
+should be failed. Setting <code>max_failures</code>to <code>0</code> means that this
+process will keep retrying until a successful (zero) exit status is
+achieved. Retries happen at most once every <code>min_duration</code> seconds
+to prevent effectively mounting a denial of service attack against
+the coordinating scheduler.</p></li>
+<li><p><code>daemon</code>: Defaulting to <code>False</code>, if <code>daemon</code> is set to <code>True</code>, a
+successful (zero) exit status does not prevent future process runs.
+Instead, the <code>Process</code> reinvokes after <code>min_duration</code> seconds.
+However, the maximum failure limit (<code>max_failures</code>) still
+applies. A combination of <code>daemon=True</code> and <code>max_failures=0</code> retries
+a <code>Process</code> indefinitely regardless of exit status. This should
+generally be avoided for very short-lived processes because of the
+accumulation of checkpointed state for each process run. When
+running in Aurora, <code>max_failures</code> is capped at
+100.</p></li>
+<li><p><code>ephemeral</code>: Defaulting to <code>False</code>, if <code>ephemeral</code> is <code>True</code>, the
+<code>Process</code>&lsquo; status is not used to determine if its bound <code>Task</code> has
+completed. For example, consider a <code>Task</code> with a
+non-ephemeral webserver process and an ephemeral logsaver process
+that periodically checkpoints its log files to a centralized data
+store. The <code>Task</code> is considered finished once the webserver process
+finishes, regardless of the logsaver&rsquo;s current status.</p></li>
+<li><p><code>min_duration</code>: Defaults to <code>15</code>. Processes may succeed or fail
+multiple times during a single Task. Each result is called a
+<em>process run</em> and this value is the minimum number of seconds the
+scheduler waits before re-running the same process.</p></li>
+<li><p><code>final</code>: Defaulting to <code>False</code>, this is a finalizing <code>Process</code> that
+should run last. Processes can be grouped into two classes:
+<em>ordinary</em> and <em>finalizing</em>. By default, Thermos Processes are
+ordinary. They run as long as the <code>Task</code> is considered
+healthy (i.e. hasn&rsquo;t reached a failure limit). But once all regular
+Thermos Processes have either finished or the <code>Task</code> has reached a
+certain failure threshold, Thermos moves into a <em>finalization</em> stage
+and runs all finalizing Processes. These are typically necessary for
+cleaning up after the <code>Task</code>, such as log checkpointers, or perhaps
+e-mail notifications of a completed Task. Finalizing processes may
+not depend upon ordinary processes or vice-versa, however finalizing
+processes may depend upon other finalizing processes and will
+otherwise run as a typical process schedule.</p></li>
+</ul>
+
+<h2 id="getting-your-code-into-the-sandbox">Getting Your Code Into The Sandbox</h2>
+
+<p>When using Aurora, you need to get your executable code into its &ldquo;sandbox&rdquo;, specifically
+the Task sandbox where the code executes for the Processes that make up that Task.</p>
+
+<p>Each Task has a sandbox created when the Task starts and garbage
+collected when it finishes. All of a Task&rsquo;s processes run in its
+sandbox, so processes can share state by using a shared current
+working directory.</p>
+
+<p>Typically, you save this code somewhere. You then need to define a Process
+in your <code>.aurora</code> configuration file that fetches the code from that somewhere
+to where the slave can see it. For a public cloud, that can be anywhere public on
+the Internet, such as S3. For a private cloud internal storage, you need to put in
+on an accessible HDFS cluster or similar storage.</p>
+
+<p>The template for this Process is:</p>
+<pre class="highlight text">&lt;name&gt; = Process(
+  name = &#39;&lt;name&gt;&#39;
+  cmdline = &#39;&lt;command to copy and extract code archive into current working directory&gt;&#39;
+)
+</pre>
+<p>Note: Be sure the extracted code archive has an executable.</p>
+
+<h2 id="defining-task-objects">Defining Task Objects</h2>
+
+<p>Tasks are handled by Mesos. A task is a collection of processes that
+runs in a shared sandbox. It&rsquo;s the fundamental unit Aurora uses to
+schedule the datacenter; essentially what Aurora does is find places
+in the cluster to run tasks.</p>
+
+<p>The key (and required) parts of a Task are:</p>
+
+<ul>
+<li><p><code>name</code>: A string giving the Task&rsquo;s name. By default, if a Task is
+not given a name, it inherits the first name in its Process list.</p></li>
+<li><p><code>processes</code>: An unordered list of Process objects bound to the Task.
+The value of the optional <code>constraints</code> attribute affects the
+contents as a whole. Currently, the only constraint, <code>order</code>, determines if
+the processes run in parallel or sequentially.</p></li>
+<li><p><code>resources</code>: A <code>Resource</code> object defining the Task&rsquo;s resource
+    footprint. A <code>Resource</code> object has three attributes:
+    -   <code>cpu</code>: A Float, the fractional number of cores the Task
+    requires.
+    -   <code>ram</code>: An Integer, RAM bytes the Task requires.
+    -   <code>disk</code>: An integer, disk bytes the Task requires.</p></li>
+</ul>
+
+<p>A basic Task definition looks like:</p>
+<pre class="highlight text">Task(
+    name=&quot;hello_world&quot;,
+    processes=[Process(name = &quot;hello_world&quot;, cmdline = &quot;echo hello world&quot;)],
+    resources=Resources(cpu = 1.0,
+                        ram = 1*GB,
+                        disk = 1*GB))
+</pre>
+<p>There are four optional Task attributes:</p>
+
+<ul>
+<li><p><code>constraints</code>: A list of <code>Constraint</code> objects that constrain the
+Task&rsquo;s processes. Currently there is only one type, the <code>order</code>
+constraint. For example the following requires that the processes
+run in the order <code>foo</code>, then <code>bar</code>.</p>
+<pre class="highlight text">constraints = [Constraint(order=[&#39;foo&#39;, &#39;bar&#39;])]
+</pre>
+<p>There is an <code>order()</code> function that takes <code>order(&#39;foo&#39;, &#39;bar&#39;, &#39;baz&#39;)</code>
+and converts it into <code>[Constraint(order=[&#39;foo&#39;, &#39;bar&#39;, &#39;baz&#39;])]</code>.
+<code>order()</code> accepts Process name strings <code>(&#39;foo&#39;, &#39;bar&#39;)</code> or the processes
+themselves, e.g. <code>foo=Process(name=&#39;foo&#39;, ...)</code>, <code>bar=Process(name=&#39;bar&#39;, ...)</code>,
+<code>constraints=order(foo, bar)</code></p>
+
+<p>Note that Thermos rejects tasks with process cycles.</p></li>
+<li><p><code>max_failures</code>: Defaulting to <code>1</code>, the number of failed processes
+needed for the <code>Task</code> to be marked as failed. Note how this
+interacts with individual Processes&rsquo; <code>max_failures</code> values. Assume a
+Task has two Processes and a <code>max_failures</code> value of <code>2</code>. So both
+Processes must fail for the Task to fail. Now, assume each of the
+Task&rsquo;s Processes has its own <code>max_failures</code> value of <code>10</code>. If
+Process &ldquo;A&rdquo; fails 5 times before succeeding, and Process &ldquo;B&rdquo; fails
+10 times and is then marked as failing, their parent Task succeeds.
+Even though there were 15 individual failures by its Processes, only
+1 of its Processes was finally marked as failing. Since 1 is less
+than the 2 that is the Task&rsquo;s <code>max_failures</code> value, the Task does
+not fail.</p></li>
+<li><p><code>max_concurrency</code>: Defaulting to <code>0</code>, the maximum number of
+concurrent processes in the Task. <code>0</code> specifies unlimited
+concurrency. For Tasks with many expensive but otherwise independent
+processes, you can limit the amount of concurrency Thermos schedules
+instead of artificially constraining them through <code>order</code>
+constraints. For example, a test framework may generate a Task with
+100 test run processes, but runs it in a Task with
+<code>resources.cpus=4</code>. Limit the amount of parallelism to 4 by setting
+<code>max_concurrency=4</code>.</p></li>
+<li><p><code>finalization_wait</code>: Defaulting to <code>30</code>, the number of seconds
+allocated for finalizing the Task&rsquo;s processes. A Task starts in
+<code>ACTIVE</code> state when Processes run and stays there as long as the Task
+is healthy and Processes run. When all Processes finish successfully
+or the Task reaches its maximum process failure limit, it goes into
+<code>CLEANING</code> state. In <code>CLEANING</code>, it sends <code>SIGTERMS</code> to any still running
+Processes. When all Processes terminate, the Task goes into
+<code>FINALIZING</code> state and invokes the schedule of all processes whose
+final attribute has a True value. Everything from the end of <code>ACTIVE</code>
+to the end of <code>FINALIZING</code> must happen within <code>finalization_wait</code>
+number of seconds. If not, all still running Processes are sent
+<code>SIGKILL</code>s (or if dependent on yet to be completed Processes, are
+never invoked).</p></li>
+</ul>
+
+<h3 id="sequentialtask:-running-processes-in-parallel-or-sequentially">SequentialTask: Running Processes in Parallel or Sequentially</h3>
+
+<p>By default, a Task with several Processes runs them in parallel. There
+are two ways to run Processes sequentially:</p>
+
+<ul>
+<li><p>Include an <code>order</code> constraint in the Task definition&rsquo;s <code>constraints</code>
+attribute whose arguments specify the processes&#39; run order:</p>
+<pre class="highlight text">Task( ... processes=[process1, process2, process3],
+      constraints = order(process1, process2, process3), ...)
+</pre></li>
+<li><p>Use <code>SequentialTask</code> instead of <code>Task</code>; it automatically runs
+processes in the order specified in the <code>processes</code> attribute. No
+<code>constraint</code> parameter is needed:</p>
+<pre class="highlight text">SequentialTask( ... processes=[process1, process2, process3] ...)
+</pre></li>
+</ul>
+
+<h3 id="simpletask">SimpleTask</h3>
+
+<p>For quickly creating simple tasks, use the <code>SimpleTask</code> helper. It
+creates a basic task from a provided name and command line using a
+default set of resources. For example, in a .<code>aurora</code> configuration
+file:</p>
+<pre class="highlight text">SimpleTask(name=&quot;hello_world&quot;, command=&quot;echo hello world&quot;)
+</pre>
+<p>is equivalent to</p>
+<pre class="highlight text">Task(name=&quot;hello_world&quot;,
+     processes=[Process(name = &quot;hello_world&quot;, cmdline = &quot;echo hello world&quot;)],
+     resources=Resources(cpu = 1.0,
+                         ram = 1*GB,
+                         disk = 1*GB))
+</pre>
+<p>The simplest idiomatic Job configuration thus becomes:</p>
+<pre class="highlight text">import os
+hello_world_job = Job(
+  task=SimpleTask(name=&quot;hello_world&quot;, command=&quot;echo hello world&quot;),
+  role=os.getenv(&#39;USER&#39;),
+  cluster=&quot;cluster1&quot;)
+</pre>
+<p>When written to <code>hello_world.aurora</code>, you invoke it with a simple
+<code>aurora create cluster1/$USER/test/hello_world hello_world.aurora</code>.</p>
+
+<h3 id="combining-tasks">Combining tasks</h3>
+
+<p><code>Tasks.concat</code>(synonym,<code>concat_tasks</code>) and
+<code>Tasks.combine</code>(synonym,<code>combine_tasks</code>) merge multiple Task definitions
+into a single Task. It may be easier to define complex Jobs
+as smaller constituent Tasks. But since a Job only includes a single
+Task, the subtasks must be combined before using them in a Job.
+Smaller Tasks can also be reused between Jobs, instead of having to
+repeat their definition for multiple Jobs.</p>
+
+<p>With both methods, the merged Task takes the first Task&rsquo;s name. The
+difference between the two is the result Task&rsquo;s process ordering.</p>
+
+<ul>
+<li><p><code>Tasks.combine</code> runs its subtasks&#39; processes in no particular order.
+The new Task&rsquo;s resource consumption is the sum of all its subtasks&#39;
+consumption.</p></li>
+<li><p><code>Tasks.concat</code> runs its subtasks in the order supplied, with each
+subtask&rsquo;s processes run serially between tasks. It is analogous to
+the <code>order</code> constraint helper, except at the Task level instead of
+the Process level. The new Task&rsquo;s resource consumption is the
+maximum value specified by any subtask for each Resource attribute
+(cpu, ram and disk).</p></li>
+</ul>
+
+<p>For example, given the following:</p>
+<pre class="highlight text">setup_task = Task(
+  ...
+  processes=[download_interpreter, update_zookeeper],
+  # It is important to note that {{Tasks.concat}} has
+  # no effect on the ordering of the processes within a task;
+  # hence the necessity of the {{order}} statement below
+  # (otherwise, the order in which {{download_interpreter}}
+  # and {{update_zookeeper}} run will be non-deterministic)
+  constraints=order(download_interpreter, update_zookeeper),
+  ...
+)
+
+run_task = SequentialTask(
+  ...
+  processes=[download_application, start_application],
+  ...
+)
+
+combined_task = Tasks.concat(setup_task, run_task)
+</pre>
+<p>The <code>Tasks.concat</code> command merges the two Tasks into a single Task and
+ensures all processes in <code>setup_task</code> run before the processes
+in <code>run_task</code>. Conceptually, the task is reduced to:</p>
+<pre class="highlight text">task = Task(
+  ...
+  processes=[download_interpreter, update_zookeeper,
+             download_application, start_application],
+  constraints=order(download_interpreter, update_zookeeper,
+                    download_application, start_application),
+  ...
+)
+</pre>
+<p>In the case of <code>Tasks.combine</code>, the two schedules run in parallel:</p>
+<pre class="highlight text">task = Task(
+  ...
+  processes=[download_interpreter, update_zookeeper,
+             download_application, start_application],
+  constraints=order(download_interpreter, update_zookeeper) +
+                    order(download_application, start_application),
+  ...
+)
+</pre>
+<p>In the latter case, each of the two sequences may operate in parallel.
+Of course, this may not be the intended behavior (for example, if
+the <code>start_application</code> Process implicitly relies
+upon <code>download_interpreter</code>). Make sure you understand the difference
+between using one or the other.</p>
+
+<h2 id="defining-job-objects">Defining Job Objects</h2>
+
+<p>A job is a group of identical tasks that Aurora can run in a Mesos cluster.</p>
+
+<p>A <code>Job</code> object is defined by the values of several attributes, some
+required and some optional. The required attributes are:</p>
+
+<ul>
+<li><p><code>task</code>: Task object to bind to this job. Note that a Job can
+only take a single Task.</p></li>
+<li><p><code>role</code>: Job&rsquo;s role account; in other words, the user account to run
+the job as on a Mesos cluster machine. A common value is
+<code>os.getenv(&#39;USER&#39;)</code>; using a Python command to get the user who
+submits the job request. The other common value is the service
+account that runs the job, e.g. <code>www-data</code>.</p></li>
+<li><p><code>environment</code>: Job&rsquo;s environment, typical values
+are <code>devel</code>, <code>test</code>, or <code>prod</code>.</p></li>
+<li><p><code>cluster</code>: Aurora cluster to schedule the job in, defined in
+<code>/etc/aurora/clusters.json</code> or <code>~/.clusters.json</code>. You can specify
+jobs where the only difference is the <code>cluster</code>, then at run time
+only run the Job whose job key includes your desired cluster&rsquo;s name.</p></li>
+</ul>
+
+<p>You usually see a <code>name</code> parameter. By default, <code>name</code> inherits its
+value from the Job&rsquo;s associated Task object, but you can override this
+default. For these four parameters, a Job definition might look like:</p>
+<pre class="highlight text">foo_job = Job( name = &#39;foo&#39;, cluster = &#39;cluster1&#39;,
+          role = os.getenv(&#39;USER&#39;), environment = &#39;prod&#39;,
+          task = foo_task)
+</pre>
+<p>In addition to the required attributes, there are several optional
+attributes. The first (strongly recommended) optional attribute is:</p>
+
+<ul>
+<li>  <code>contact</code>: An email address for the Job&rsquo;s owner. For production
+jobs, it is usually a team mailing list.</li>
+</ul>
+
+<p>Two more attributes deal with how to handle failure of the Job&rsquo;s Task:</p>
+
+<ul>
+<li><p><code>max_task_failures</code>: An integer, defaulting to <code>1</code>, of the maximum
+number of Task failures after which the Job is considered failed.
+<code>-1</code> allows for infinite failures.</p></li>
+<li><p><code>service</code>: A boolean, defaulting to <code>False</code>, which if <code>True</code>
+restarts tasks regardless of whether they succeeded or failed. In
+other words, if <code>True</code>, after the Job&rsquo;s Task completes, it
+automatically starts again. This is for Jobs you want to run
+continuously, rather than doing a single run.</p></li>
+</ul>
+
+<p>Three attributes deal with configuring the Job&rsquo;s Task:</p>
+
+<ul>
+<li><p><code>instances</code>: Defaulting to <code>1</code>, the number of
+instances/replicas/shards of the Job&rsquo;s Task to create.</p></li>
+<li><p><code>priority</code>: Defaulting to <code>0</code>, the Job&rsquo;s Task&rsquo;s preemption priority,
+for which higher values may preempt Tasks from Jobs with lower
+values.</p></li>
+<li><p><code>production</code>: a Boolean, defaulting to <code>False</code>, specifying that this
+is a production job backed by quota. Tasks from production Jobs may
+preempt tasks from any non-production job, and may only be preempted
+by tasks from production jobs in the same role with higher
+priority. <strong>WARNING</strong>: To run Jobs at this level, the Job role must
+have the appropriate quota.</p></li>
+</ul>
+
+<p>The final three Job attributes each take an object as their value.</p>
+
+<ul>
+<li>  <code>update_config</code>: An <code>UpdateConfig</code>
+object provides parameters for controlling the rate and policy of
+rolling updates. The <code>UpdateConfig</code> parameters are:
+
+<ul>
+<li>  <code>batch_size</code>: An integer, defaulting to <code>1</code>, specifying the
+maximum number of shards to update in one iteration.</li>
+<li>  <code>restart_threshold</code>: An integer, defaulting to <code>60</code>, specifying
+the maximum number of seconds before a shard must move into the
+<code>RUNNING</code> state before considered a failure.</li>
+<li>  <code>watch_secs</code>: An integer, defaulting to <code>45</code>, specifying the
+minimum number of seconds a shard must remain in the <code>RUNNING</code>
+state before considered a success.</li>
+<li>  <code>max_per_shard_failures</code>: An integer, defaulting to <code>0</code>,
+specifying the maximum number of restarts per shard during an
+update. When the limit is exceeded, it increments the total
+failure count.</li>
+<li>  <code>max_total_failures</code>: An integer, defaulting to <code>0</code>, specifying
+the maximum number of shard failures tolerated during an update.
+Cannot be equal to or greater than the job&rsquo;s total number of
+tasks.</li>
+</ul></li>
+<li>  <code>health_check_config</code>: A <code>HealthCheckConfig</code> object that provides
+parameters for controlling a Task&rsquo;s health checks via HTTP. Only
+used if a health port was assigned with a command line wildcard. The
+<code>HealthCheckConfig</code> parameters are:
+
+<ul>
+<li>  <code>initial_interval_secs</code>: An integer, defaulting to <code>15</code>,
+specifying the initial delay for doing an HTTP health check.</li>
+<li>  <code>interval_secs</code>: An integer, defaulting to <code>10</code>, specifying the
+number of seconds in the interval between checking the Task&rsquo;s
+health.</li>
+<li>  <code>timeout_secs</code>: An integer, defaulting to <code>1</code>, specifying the
+number of seconds the application must respond to an HTTP health
+check with <code>OK</code> before it is considered a failure.</li>
+<li>  <code>max_consecutive_failures</code>: An integer, defaulting to <code>0</code>,
+specifying the maximum number of consecutive failures before a
+task is unhealthy.</li>
+</ul></li>
+<li>  <code>constraints</code>: A <code>dict</code> Python object, specifying Task scheduling
+constraints. Most users will not need to specify constraints, as the
+scheduler automatically inserts reasonable defaults. Please do not
+set this field unless you are sure of what you are doing. See the
+section in the Aurora + Thermos Reference manual on <a href="/documentation/latest/configuration-reference/">Specifying
+Scheduling Constraints</a> for more information.</li>
+</ul>
+
+<h2 id="the-jobs-list">The jobs List</h2>
+
+<p>At the end of your <code>.aurora</code> file, you need to specify a list of the
+file&rsquo;s defined Jobs to run in the order listed. For example, the
+following runs first <code>job1</code>, then <code>job2</code>, then <code>job3</code>.</p>
+
+<p>jobs = [job1, job2, job3]</p>
+
+<h2 id="templating">Templating</h2>
+
+<p>The <code>.aurora</code> file format is just Python. However, <code>Job</code>, <code>Task</code>,
+<code>Process</code>, and other classes are defined by a templating library called
+<em>Pystachio</em>, a powerful tool for configuration specification and reuse.</p>
+
+<p><a href="/documentation/latest/configuration-reference/">Aurora+Thermos Configuration Reference</a>
+has a full reference of all Aurora/Thermos defined Pystachio objects.</p>
+
+<p>When writing your <code>.aurora</code> file, you may use any Pystachio datatypes, as
+well as any objects shown in the <em>Aurora+Thermos Configuration
+Reference</em> without <code>import</code> statements - the Aurora config loader
+injects them automatically. Other than that the <code>.aurora</code> format
+works like any other Python script.</p>
+
+<h3 id="templating-1:-binding-in-pystachio">Templating 1: Binding in Pystachio</h3>
+
+<p>Pystachio uses the visually distinctive {{}} to indicate template
+variables. These are often called &ldquo;mustache variables&rdquo; after the
+similarly appearing variables in the Mustache templating system and
+because the curly braces resemble mustaches.</p>
+
+<p>If you are familiar with the Mustache system, templates in Pystachio
+have significant differences. They have no nesting, joining, or
+inheritance semantics. On the other hand, when evaluated, templates
+are evaluated iteratively, so this affords some level of indirection.</p>
+
+<p>Let&rsquo;s start with the simplest template; text with one
+variable, in this case <code>name</code>;</p>
+<pre class="highlight text">Hello {{name}}
+</pre>
+<p>If we evaluate this as is, we&rsquo;d get back:</p>
+<pre class="highlight text">Hello
+</pre>
+<p>If a template variable doesn&rsquo;t have a value, when evaluated it&rsquo;s
+replaced with nothing. If we add a binding to give it a value:</p>
+<pre class="highlight json"><span class="p">{</span><span class="w"> </span><span class="s2">&quot;name&quot;</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="s2">&quot;Tom&quot;</span><span class="w"> </span><span class="p">}</span><span class="w">
+</span></pre>
+<p>We&rsquo;d get back:</p>
+<pre class="highlight text">Hello Tom
+</pre>
+<p>We can also use {{}} variables as sectional variables. Let&rsquo;s say we
+have:</p>
+<pre class="highlight text">{{#x}} Testing... {{/x}}
+</pre>
+<p>If <code>x</code> evaluates to <code>True</code>, the text between the sectional tags is
+shown. If there is no value for <code>x</code> or it evaluates to <code>False</code>, the
+between tags text is not shown. So, at a basic level, a sectional
+variable acts as a conditional.</p>
+
+<p>However, if the sectional variable evaluates to a list, array, etc. it
+acts as a <code>foreach</code>. For example,</p>
+<pre class="highlight text">{{#x}} {{name}} {{/x}}
+</pre>
+<p>with</p>
+<pre class="highlight json"><span class="p">{</span><span class="w"> </span><span class="s2">&quot;x&quot;</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="s2">&quot;name&quot;</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="s2">&quot;tic&quot;</span><span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="s2">&quot;name&quot;</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="s2">&quot;tac&quot;</span><span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="s2">&quot;name&quot;</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="s2">&quot;toe&quot;</span><span class="w"> </span><span class="p">}</span
 ><span class="w"> </span><span class="p">]</span><span class="w"> </span><span class="p">}</span><span class="w">
+</span></pre>
+<p>evaluates to</p>
+<pre class="highlight text">tic tac toe
+</pre>
+<p>Every Pystachio object has an associated <code>.bind</code> method that can bind
+values to {{}} variables. Bindings are not immediately evaluated.
+Instead, they are evaluated only when the interpolated value of the
+object is necessary, e.g. for performing equality or serializing a
+message over the wire.</p>
+
+<p>Objects with and without mustache templated variables behave
+differently:</p>
+<pre class="highlight text">&gt;&gt;&gt; Float(1.5)
+Float(1.5)
+
+&gt;&gt;&gt; Float(&#39;{{x}}.5&#39;)
+Float({{x}}.5)
+
+&gt;&gt;&gt; Float(&#39;{{x}}.5&#39;).bind(x = 1)
+Float(1.5)
+
+&gt;&gt;&gt; Float(&#39;{{x}}.5&#39;).bind(x = 1) == Float(1.5)
+True
+
+&gt;&gt;&gt; contextual_object = String(&#39;{{metavar{{number}}}}&#39;).bind(
+... metavar1 = &quot;first&quot;, metavar2 = &quot;second&quot;)
+
+&gt;&gt;&gt; contextual_object
+String({{metavar{{number}}}})
+
+&gt;&gt;&gt; contextual_object.bind(number = 1)
+String(first)
+
+&gt;&gt;&gt; contextual_object.bind(number = 2)
+String(second)
+</pre>
+<p>You usually bind simple key to value pairs, but you can also bind three
+other objects: lists, dictionaries, and structurals. These will be
+described in detail later.</p>
+
+<h3 id="structurals-in-pystachio-/-aurora">Structurals in Pystachio / Aurora</h3>
+
+<p>Most Aurora/Thermos users don&rsquo;t ever (knowingly) interact with <code>String</code>,
+<code>Float</code>, or <code>Integer</code> Pystashio objects directly. Instead they interact
+with derived structural (<code>Struct</code>) objects that are collections of
+fundamental and structural objects. The structural object components are
+called <em>attributes</em>. Aurora&rsquo;s most used structural objects are <code>Job</code>,
+<code>Task</code>, and <code>Process</code>:</p>
+<pre class="highlight text">class Process(Struct):
+  cmdline = Required(String)
+  name = Required(String)
+  max_failures = Default(Integer, 1)
+  daemon = Default(Boolean, False)
+  ephemeral = Default(Boolean, False)
+  min_duration = Default(Integer, 5)
+  final = Default(Boolean, False)
+</pre>
+<p>Construct default objects by following the object&rsquo;s type with (). If you
+want an attribute to have a value different from its default, include
+the attribute name and value inside the parentheses.</p>
+<pre class="highlight text">&gt;&gt;&gt; Process()
+Process(daemon=False, max_failures=1, ephemeral=False,
+  min_duration=5, final=False)
+</pre>
+<p>Attribute values can be template variables, which then receive specific
+values when creating the object.</p>
+<pre class="highlight text">&gt;&gt;&gt; Process(cmdline = &#39;echo {{message}}&#39;)
+Process(daemon=False, max_failures=1, ephemeral=False, min_duration=5,
+        cmdline=echo {{message}}, final=False)
+
+&gt;&gt;&gt; Process(cmdline = &#39;echo {{message}}&#39;).bind(message = &#39;hello world&#39;)
+Process(daemon=False, max_failures=1, ephemeral=False, min_duration=5,
+        cmdline=echo hello world, final=False)
+</pre>
+<p>A powerful binding property is that all of an object&rsquo;s children inherit its
+bindings:</p>
+<pre class="highlight text">&gt;&gt;&gt; List(Process)([
+... Process(name = &#39;{{prefix}}_one&#39;),
+... Process(name = &#39;{{prefix}}_two&#39;)
+... ]).bind(prefix = &#39;hello&#39;)
+ProcessList(
+  Process(daemon=False, name=hello_one, max_failures=1, ephemeral=False, min_duration=5, final=False),
+  Process(daemon=False, name=hello_two, max_failures=1, ephemeral=False, min_duration=5, final=False)
+  )
+</pre>
+<p>Remember that an Aurora Job contains Tasks which contain Processes. A
+Job level binding is inherited by its Tasks and all their Processes.
+Similarly a Task level binding is available to that Task and its
+Processes but is <em>not</em> visible at the Job level (inheritance is a
+one-way street.)</p>
+
+<h4 id="mustaches-within-structurals">Mustaches Within Structurals</h4>
+
+<p>When you define a <code>Struct</code> schema, one powerful, but confusing, feature
+is that all of that structure&rsquo;s attributes are Mustache variables within
+the enclosing scope <em>once they have been populated</em>.</p>
+
+<p>For example, when <code>Process</code> is defined above, all its attributes such as
+{{<code>name</code>}}, {{<code>cmdline</code>}}, {{<code>max_failures</code>}} etc., are all immediately
+defined as Mustache variables, implicitly bound into the <code>Process</code>, and
+inherit all child objects once they are defined.</p>
+
+<p>Thus, you can do the following:</p>
+<pre class="highlight text">&gt;&gt;&gt; Process(name = &quot;installer&quot;, cmdline = &quot;echo {{name}} is running&quot;)
+Process(daemon=False, name=installer, max_failures=1, ephemeral=False, min_duration=5,
+        cmdline=echo installer is running, final=False)
+</pre>
+<p>WARNING: This binding only takes place in one direction. For example,
+the following does NOT work and does not set the <code>Process</code> <code>name</code>
+attribute&rsquo;s value.</p>
+<pre class="highlight text">&gt;&gt;&gt; Process().bind(name = &quot;installer&quot;)
+Process(daemon=False, max_failures=1, ephemeral=False, min_duration=5, final=False)
+</pre>
+<p>The following is also not possible and results in an infinite loop that
+attempts to resolve <code>Process.name</code>.</p>
+<pre class="highlight text">&gt;&gt;&gt; Process(name = &#39;{{name}}&#39;).bind(name = &#39;installer&#39;)
+</pre>
+<p>Do not confuse Structural attributes with bound Mustache variables.
+Attributes are implicitly converted to Mustache variables but not vice
+versa.</p>
+
+<h3 id="templating-2:-structurals-are-factories">Templating 2: Structurals Are Factories</h3>
+
+<h4 id="a-second-way-of-templating">A Second Way of Templating</h4>
+
+<p>A second templating method is both as powerful as the aforementioned and
+often confused with it. This method is due to automatic conversion of
+Struct attributes to Mustache variables as described above.</p>
+
+<p>Suppose you create a Process object:</p>
+<pre class="highlight text">&gt;&gt;&gt; p = Process(name = &quot;process_one&quot;, cmdline = &quot;echo hello world&quot;)
+
+&gt;&gt;&gt; p
+Process(daemon=False, name=process_one, max_failures=1, ephemeral=False, min_duration=5,
+        cmdline=echo hello world, final=False)
+</pre>
+<p>This <code>Process</code> object, &ldquo;<code>p</code>&rdquo;, can be used wherever a <code>Process</code> object is
+needed. It can also be reused by changing the value(s) of its
+attribute(s). Here we change its <code>name</code> attribute from <code>process_one</code> to
+<code>process_two</code>.</p>
+<pre class="highlight text">&gt;&gt;&gt; p(name = &quot;process_two&quot;)
+Process(daemon=False, name=process_two, max_failures=1, ephemeral=False, min_duration=5,
+        cmdline=echo hello world, final=False)
+</pre>
+<p>Template creation is a common use for this technique:</p>
+<pre class="highlight text">&gt;&gt;&gt; Daemon = Process(daemon = True)
+&gt;&gt;&gt; logrotate = Daemon(name = &#39;logrotate&#39;, cmdline = &#39;./logrotate conf/logrotate.conf&#39;)
+&gt;&gt;&gt; mysql = Daemon(name = &#39;mysql&#39;, cmdline = &#39;bin/mysqld --safe-mode&#39;)
+</pre>
+<h3 id="advanced-binding">Advanced Binding</h3>
+
+<p>As described above, <code>.bind()</code> binds simple strings or numbers to
+Mustache variables. In addition to Structural types formed by combining
+atomic types, Pystachio has two container types; <code>List</code> and <code>Map</code> which
+can also be bound via <code>.bind()</code>.</p>
+
+<h4 id="bind-syntax">Bind Syntax</h4>
+
+<p>The <code>bind()</code> function can take Python dictionaries or <code>kwargs</code>
+interchangeably (when &ldquo;<code>kwargs</code>&rdquo; is in a function definition, <code>kwargs</code>
+receives a Python dictionary containing all keyword arguments after the
+formal parameter list).</p>
+<pre class="highlight text">&gt;&gt;&gt; String(&#39;{{foo}}&#39;).bind(foo = &#39;bar&#39;) == String(&#39;{{foo}}&#39;).bind({&#39;foo&#39;: &#39;bar&#39;})
+True
+</pre>
+<p>Bindings done &ldquo;closer&rdquo; to the object in question take precedence:</p>
+<pre class="highlight text">&gt;&gt;&gt; p = Process(name = &#39;{{context}}_process&#39;)
+&gt;&gt;&gt; t = Task().bind(context = &#39;global&#39;)
+&gt;&gt;&gt; t(processes = [p, p.bind(context = &#39;local&#39;)])
+Task(processes=ProcessList(
+  Process(daemon=False, name=global_process, max_failures=1, ephemeral=False, final=False,
+          min_duration=5),
+  Process(daemon=False, name=local_process, max_failures=1, ephemeral=False, final=False,
+          min_duration=5)
+))
+</pre>
+<h4 id="binding-complex-objects">Binding Complex Objects</h4>
+
+<h5 id="lists">Lists</h5>
+<pre class="highlight text">&gt;&gt;&gt; fibonacci = List(Integer)([1, 1, 2, 3, 5, 8, 13])
+&gt;&gt;&gt; String(&#39;{{fib[4]}}&#39;).bind(fib = fibonacci)
+String(5)
+</pre>
+<h5 id="maps">Maps</h5>
+<pre class="highlight text">&gt;&gt;&gt; first_names = Map(String, String)({&#39;Kent&#39;: &#39;Clark&#39;, &#39;Wayne&#39;: &#39;Bruce&#39;, &#39;Prince&#39;: &#39;Diana&#39;})
+&gt;&gt;&gt; String(&#39;{{first[Kent]}}&#39;).bind(first = first_names)
+String(Clark)
+</pre>
+<h5 id="structurals">Structurals</h5>
+<pre class="highlight text">&gt;&gt;&gt; String(&#39;{{p.cmdline}}&#39;).bind(p = Process(cmdline = &quot;echo hello world&quot;))
+String(echo hello world)
+</pre>
+<h3 id="structural-binding">Structural Binding</h3>
+
+<p>Use structural templates when binding more than two or three individual
+values at the Job or Task level. For fewer than two or three, standard
+key to string binding is sufficient.</p>
+
+<p>Structural binding is a very powerful pattern and is most useful in
+Aurora/Thermos for doing Structural configuration. For example, you can
+define a job profile. The following profile uses <code>HDFS</code>, the Hadoop
+Distributed File System, to designate a file&rsquo;s location. <code>HDFS</code> does
+not come with Aurora, so you&rsquo;ll need to either install it separately
+or change the way the dataset is designated.</p>
+<pre class="highlight text">class Profile(Struct):
+  version = Required(String)
+  environment = Required(String)
+  dataset = Default(String, hdfs://home/aurora/data/{{environment}}&#39;)
+
+PRODUCTION = Profile(version = &#39;live&#39;, environment = &#39;prod&#39;)
+DEVEL = Profile(version = &#39;latest&#39;,
+                environment = &#39;devel&#39;,
+                dataset = &#39;hdfs://home/aurora/data/test&#39;)
+TEST = Profile(version = &#39;latest&#39;, environment = &#39;test&#39;)
+
+JOB_TEMPLATE = Job(
+  name = &#39;application&#39;,
+  role = &#39;myteam&#39;,
+  cluster = &#39;cluster1&#39;,
+  environment = &#39;{{profile.environment}}&#39;,
+  task = SequentialTask(
+    name = &#39;task&#39;,
+    resources = Resources(cpu = 2, ram = 4*GB, disk = 8*GB),
+    processes = [
+  Process(name = &#39;main&#39;, cmdline = &#39;java -jar application.jar -hdfsPath
+             {{profile.dataset}}&#39;)
+    ]
+   )
+ )
+
+jobs = [
+  JOB_TEMPLATE(instances = 100).bind(profile = PRODUCTION),
+  JOB_TEMPLATE.bind(profile = DEVEL),
+  JOB_TEMPLATE.bind(profile = TEST),
+ ]
+</pre>
+<p>In this case, a custom structural &ldquo;Profile&rdquo; is created to self-document
+the configuration to some degree. This also allows some schema
+&ldquo;type-checking&rdquo;, and for default self-substitution, e.g. in
+<code>Profile.dataset</code> above.</p>
+
+<p>So rather than a <code>.bind()</code> with a half-dozen substituted variables, you
+can bind a single object that has sensible defaults stored in a single
+place.</p>
+
+<h2 id="configuration-file-writing-tips-and-best-practices">Configuration File Writing Tips And Best Practices</h2>
+
+<h3 id="use-as-few-.aurora-files-as-possible">Use As Few .aurora Files As Possible</h3>
+
+<p>When creating your <code>.aurora</code> configuration, try to keep all versions of
+a particular job within the same <code>.aurora</code> file. For example, if you
+have separate jobs for <code>cluster1</code>, <code>cluster1</code> staging, <code>cluster1</code>
+testing, and<code>cluster2</code>, keep them as close together as possible.</p>
+
+<p>Constructs shared across multiple jobs owned by your team (e.g.
+team-level defaults or structural templates) can be split into separate
+<code>.aurora</code>files and included via the <code>include</code> directive.</p>
+
+<h3 id="avoid-boilerplate">Avoid Boilerplate</h3>
+
+<p>If you see repetition or find yourself copy and pasting any parts of
+your configuration, it&rsquo;s likely an opportunity for templating. Take the
+example below:</p>
+
+<p><code>redundant.aurora</code> contains:</p>
+<pre class="highlight text">download = Process(
+  name = &#39;download&#39;,
+  cmdline = &#39;wget http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2&#39;,
+  max_failures = 5,
+  min_duration = 1)
+
+unpack = Process(
+  name = &#39;unpack&#39;,
+  cmdline = &#39;rm -rf Python-2.7.3 &amp;&amp; tar xzf Python-2.7.3.tar.bz2&#39;,
+  max_failures = 5,
+  min_duration = 1)
+
+build = Process(
+  name = &#39;build&#39;,
+  cmdline = &#39;pushd Python-2.7.3 &amp;&amp; ./configure &amp;&amp; make &amp;&amp; popd&#39;,
+  max_failures = 1)
+
+email = Process(
+  name = &#39;email&#39;,
+  cmdline = &#39;echo Success | mail feynman@tmc.com&#39;,
+  max_failures = 5,
+  min_duration = 1)
+
+build_python = Task(
+  name = &#39;build_python&#39;,
+  processes = [download, unpack, build, email],
+  constraints = [Constraint(order = [&#39;download&#39;, &#39;unpack&#39;, &#39;build&#39;, &#39;email&#39;])])
+</pre>
+<p>As you&rsquo;ll notice, there&rsquo;s a lot of repetition in the <code>Process</code>
+definitions. For example, almost every process sets a <code>max_failures</code>
+limit to 5 and a <code>min_duration</code> to 1. This is an opportunity for factoring
+into a common process template.</p>
+
+<p>Furthermore, the Python version is repeated everywhere. This can be
+bound via structural templating as described in the <a href="#AdvancedBinding">Advanced Binding</a>
+section.</p>
+
+<p><code>less_redundant.aurora</code> contains:</p>
+<pre class="highlight text">class Python(Struct):
+  version = Required(String)
+  base = Default(String, &#39;Python-{{version}}&#39;)
+  package = Default(String, &#39;{{base}}.tar.bz2&#39;)
+
+ReliableProcess = Process(
+  max_failures = 5,
+  min_duration = 1)
+
+download = ReliableProcess(
+  name = &#39;download&#39;,
+  cmdline = &#39;wget http://www.python.org/ftp/python/{{python.version}}/{{python.package}}&#39;)
+
+unpack = ReliableProcess(
+  name = &#39;unpack&#39;,
+  cmdline = &#39;rm -rf {{python.base}} &amp;&amp; tar xzf {{python.package}}&#39;)
+
+build = ReliableProcess(
+  name = &#39;build&#39;,
+  cmdline = &#39;pushd {{python.base}} &amp;&amp; ./configure &amp;&amp; make &amp;&amp; popd&#39;,
+  max_failures = 1)
+
+email = ReliableProcess(
+  name = &#39;email&#39;,
+  cmdline = &#39;echo Success | mail {{role}}@foocorp.com&#39;)
+
+build_python = SequentialTask(
+  name = &#39;build_python&#39;,
+  processes = [download, unpack, build, email]).bind(python = Python(version = &quot;2.7.3&quot;))
+</pre>
+<h3 id="thermos-uses-bash,-but-thermos-is-not-bash">Thermos Uses bash, But Thermos Is Not bash</h3>
+
+<h4 id="bad">Bad</h4>
+
+<p>Many tiny Processes makes for harder to manage configurations.</p>
+<pre class="highlight text">copy = Process(
+  name = &#39;copy&#39;,
+  cmdline = &#39;rcp user@my_machine:my_application .&#39;
+ )
+
+ unpack = Process(
+   name = &#39;unpack&#39;,
+   cmdline = &#39;unzip app.zip&#39;
+ )
+
+ remove = Process(
+   name = &#39;remove&#39;,
+   cmdline = &#39;rm -f app.zip&#39;
+ )
+
+ run = Process(
+   name = &#39;app&#39;,
+   cmdline = &#39;java -jar app.jar&#39;
+ )
+
+ run_task = Task(
+   processes = [copy, unpack, remove, run],
+   constraints = order(copy, unpack, remove, run)
+ )
+</pre>
+<h4 id="good">Good</h4>
+
+<p>Each <code>cmdline</code> runs in a bash subshell, so you have the full power of
+bash. Chaining commands with <code>&amp;&amp;</code> or <code>||</code> is almost always the right
+thing to do.</p>
+
+<p>Also for Tasks that are simply a list of processes that run one after
+another, consider using the <code>SequentialTask</code> helper which applies a
+linear ordering constraint for you.</p>
+<pre class="highlight text">stage = Process(
+  name = &#39;stage&#39;,
+  cmdline = &#39;rcp user@my_machine:my_application . &amp;&amp; unzip app.zip &amp;&amp; rm -f app.zip&#39;)
+
+run = Process(name = &#39;app&#39;, cmdline = &#39;java -jar app.jar&#39;)
+
+run_task = SequentialTask(processes = [stage, run])
+</pre>
+<h3 id="rarely-use-functions-in-your-configurations">Rarely Use Functions In Your Configurations</h3>
+
+<p>90% of the time you define a function in a <code>.aurora</code> file, you&rsquo;re
+probably Doing It Wrong&trade;.</p>
+
+<h4 id="bad">Bad</h4>
+<pre class="highlight text">def get_my_task(name, user, cpu, ram, disk):
+  return Task(
+    name = name,
+    user = user,
+    processes = [STAGE_PROCESS, RUN_PROCESS],
+    constraints = order(STAGE_PROCESS, RUN_PROCESS),
+    resources = Resources(cpu = cpu, ram = ram, disk = disk)
+ )
+
+ task_one = get_my_task(&#39;task_one&#39;, &#39;feynman&#39;, 1.0, 32*MB, 1*GB)
+ task_two = get_my_task(&#39;task_two&#39;, &#39;feynman&#39;, 2.0, 64*MB, 1*GB)
+</pre>
+<h4 id="good">Good</h4>
+
+<p>This one is more idiomatic. Forced keyword arguments prevents accidents,
+e.g. constructing a task with &ldquo;32*MB&rdquo; when you mean 32MB of ram and not
+disk. Less proliferation of task-construction techniques means
+easier-to-read, quicker-to-understand, and a more composable
+configuration.</p>
+<pre class="highlight text">TASK_TEMPLATE = SequentialTask(
+  user = &#39;wickman&#39;,
+  processes = [STAGE_PROCESS, RUN_PROCESS],
+)
+
+task_one = TASK_TEMPLATE(
+  name = &#39;task_one&#39;,
+  resources = Resources(cpu = 1.0, ram = 32*MB, disk = 1*GB) )
+
+task_two = TASK_TEMPLATE(
+  name = &#39;task_two&#39;,
+  resources = Resources(cpu = 2.0, ram = 64*MB, disk = 1*GB)
+)
+</pre>
+  		</div>
+  	  </div>
+	  
+      	<div class="container-fluid section-footer buffer">
+      <div class="container">
+        <div class="row">
+		  <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+		  <ul>
+		    <li><a href="/downloads/">Downloads</a></li>
+            <li><a href="/community/">Mailing Lists</a></li>
+			<li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+			<li><a href="/documentation/latest/contributing/">How To Contribute</a></li>     
+		  </ul>
+	      </div>
+		  <div class="col-md-2"><h3>The ASF</h3>
+          <ul>
+            <li><a href="http://www.apache.org/licenses/">License</a></li>
+            <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>  
+            <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+            <li><a href="http://www.apache.org/security/">Security</a></li>
+          </ul>
+		  </div>
+		  <div class="col-md-6">
+		    <p class="disclaimer">Apache Aurora is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.</p>
+			<p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+        </div>
+      </div>
+    </div>
+	</body>
+</html>
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-aurora-website/blob/c43a3a2d/publish/documentation/latest/contributing/index.html
----------------------------------------------------------------------
diff --git a/publish/documentation/latest/contributing/index.html b/publish/documentation/latest/contributing/index.html
new file mode 100644
index 0000000..4a461de
--- /dev/null
+++ b/publish/documentation/latest/contributing/index.html
@@ -0,0 +1,133 @@
+<!DOCTYPE html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+	<title>Apache Aurora</title>
+    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+    <link href="/assets/css/main.css" rel="stylesheet">
+	<!-- Analytics -->
+	<script type="text/javascript">
+		  var _gaq = _gaq || [];
+		  _gaq.push(['_setAccount', 'UA-45879646-1']);
+		  _gaq.push(['_setDomainName', 'apache.org']);
+		  _gaq.push(['_trackPageview']);
+
+		  (function() {
+		    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+		    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+		    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+		  })();
+	</script>
+  </head>
+  <body>
+	  
+        <div class="container-fluid section-header">
+  <div class="container">
+    <div class="nav nav-bar">
+    <a href="/"><img src="/assets/img/aurora_logo_white_bkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+	<ul class="nav navbar-nav navbar-right">
+      <li><a href="/documentation/latest/">Documentation</a></li>
+      <li><a href="/community/">Community</a></li>
+      <li><a href="/downloads/">Downloads</a></li>
+      <li><a href="/blog/">Blog</a></li>
+    </ul>
+    </div>
+  </div>
+</div>	
+  	  <div class="container-fluid">
+  	  	<div class="container content">
+          <h2 id="get-the-source-code">Get the Source Code</h2>
+
+<p>First things first, you&rsquo;ll need the source! The Aurora source is available from Apache git:</p>
+<pre class="highlight text">git clone https://git-wip-us.apache.org/repos/asf/incubator-aurora
+</pre>
+<h2 id="find-something-to-do">Find Something to Do</h2>
+
+<p>There are issues in <a href="https://issues.apache.org/jira/browse/AURORA">Jira</a> with the
+<a href="https://issues.apache.org/jira/browse/AURORA-189?jql=project%20%3D%20AURORA%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20newbie%20ORDER%20BY%20priority%20DESC">&ldquo;newbie&rdquo; tag</a>
+that are good starting places for new Aurora contributors; pick one of these and dive in! Once
+you&rsquo;ve got a patch, the next step is to post a review.</p>
+
+<h2 id="getting-your-reviewboard-account">Getting your ReviewBoard Account</h2>
+
+<p>Go to <a href="https://reviews.apache.org">https://reviews.apache.org</a> and create an account.</p>
+
+<h2 id="setting-up-your-reviewboard-environment">Setting up your ReviewBoard Environment</h2>
+
+<p>Run <code>./rbt status</code>. The first time this runs it will bootstrap and you will be asked to login.
+Subsequent runs will cache your login credentials.</p>
+
+<h2 id="submitting-a-patch-for-review">Submitting a Patch for Review</h2>
+
+<p>Post a review with <code>rbt</code>, fill out the fields in your browser and hit Publish.</p>
+<pre class="highlight text">./rbt post -o
+</pre>
+<p>Once you&rsquo;ve done this, you probably want to mark the associated Jira issue as Reviewable.</p>
+
+<h2 id="updating-an-existing-review">Updating an Existing Review</h2>
+
+<p>Incorporate review feedback, make some more commits, update your existing review, fill out the
+fields in your browser and hit Publish.</p>
+<pre class="highlight text">./rbt post -o -r &lt;RB_ID&gt;
+</pre>
+<h2 id="merging-your-own-review-(committers)">Merging Your Own Review (Committers)</h2>
+
+<p>Once you have shipits from the right committers, merge your changes in a single commit and mark
+the review as submitted. The typical workflow is:</p>
+<pre class="highlight text">git checkout master
+git pull origin master
+./rbt patch -c &lt;RB_ID&gt;  # Verify the automatically-generated commit message looks sane,
+                        # editing if necessary.
+git show master         # Verify everything looks sane
+git push origin master
+./rbt close &lt;RB_ID&gt;
+</pre>
+<p>Note that even if you&rsquo;re developing using feature branches you will not use <code>git merge</code> - each
+commit will be an atomic change accompanied by a ReviewBoard entry.</p>
+
+<h2 id="merging-someone-else&#39;s-review">Merging Someone Else&rsquo;s Review</h2>
+
+<p>Sometimes you&rsquo;ll need to merge someone else&rsquo;s RB. The typical workflow for this is</p>
+<pre class="highlight text">git checkout master
+git pull origin master
+./rbt patch -c &lt;RB_ID&gt;
+git show master  # Verify everything looks sane, author is correct
+git push origin master
+</pre>
+<h2 id="cleaning-up">Cleaning Up</h2>
+
+<p>Your patch has landed, congratulations! The last thing you&rsquo;ll want to do before moving on to your
+next fix is to clean up your Jira and Reviewboard. The former of which should be marked as
+&ldquo;Resolved&rdquo; while the latter should be marked as &ldquo;Submitted&rdquo;.</p>
+
+  		</div>
+  	  </div>
+	  
+      	<div class="container-fluid section-footer buffer">
+      <div class="container">
+        <div class="row">
+		  <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+		  <ul>
+		    <li><a href="/downloads/">Downloads</a></li>
+            <li><a href="/community/">Mailing Lists</a></li>
+			<li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+			<li><a href="/documentation/latest/contributing/">How To Contribute</a></li>     
+		  </ul>
+	      </div>
+		  <div class="col-md-2"><h3>The ASF</h3>
+          <ul>
+            <li><a href="http://www.apache.org/licenses/">License</a></li>
+            <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>  
+            <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+            <li><a href="http://www.apache.org/security/">Security</a></li>
+          </ul>
+		  </div>
+		  <div class="col-md-6">
+		    <p class="disclaimer">Apache Aurora is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.</p>
+			<p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+        </div>
+      </div>
+    </div>
+	</body>
+</html>
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-aurora-website/blob/c43a3a2d/publish/documentation/latest/cron-jobs/index.html
----------------------------------------------------------------------
diff --git a/publish/documentation/latest/cron-jobs/index.html b/publish/documentation/latest/cron-jobs/index.html
new file mode 100644
index 0000000..e540831
--- /dev/null
+++ b/publish/documentation/latest/cron-jobs/index.html
@@ -0,0 +1,215 @@
+<!DOCTYPE html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+	<title>Apache Aurora</title>
+    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+    <link href="/assets/css/main.css" rel="stylesheet">
+	<!-- Analytics -->
+	<script type="text/javascript">
+		  var _gaq = _gaq || [];
+		  _gaq.push(['_setAccount', 'UA-45879646-1']);
+		  _gaq.push(['_setDomainName', 'apache.org']);
+		  _gaq.push(['_trackPageview']);
+
+		  (function() {
+		    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+		    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+		    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+		  })();
+	</script>
+  </head>
+  <body>
+	  
+        <div class="container-fluid section-header">
+  <div class="container">
+    <div class="nav nav-bar">
+    <a href="/"><img src="/assets/img/aurora_logo_white_bkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+	<ul class="nav navbar-nav navbar-right">
+      <li><a href="/documentation/latest/">Documentation</a></li>
+      <li><a href="/community/">Community</a></li>
+      <li><a href="/downloads/">Downloads</a></li>
+      <li><a href="/blog/">Blog</a></li>
+    </ul>
+    </div>
+  </div>
+</div>	
+  	  <div class="container-fluid">
+  	  	<div class="container content">
+          <h1 id="cron-jobs">Cron Jobs</h1>
+
+<p>Aurora supports execution of scheduled jobs on a Mesos cluster using cron-style syntax.</p>
+
+<ul>
+<li><a href="#overview">Overview</a></li>
+<li><a href="#collision-policies">Collision Policies</a>
+
+<ul>
+<li><a href="#kill_existing">KILL_EXISTING</a></li>
+<li><a href="#cancel_new">CANCEL_NEW</a></li>
+</ul></li>
+<li><a href="#failure-recovery">Failure recovery</a></li>
+<li><a href="#interacting-with-cron-jobs-via-the-aurora-cli">Interacting with cron jobs via the Aurora CLI</a>
+
+<ul>
+<li><a href="#cron-schedule">cron schedule</a></li>
+<li><a href="#cron-deschedule">cron deschedule</a></li>
+<li><a href="#cron-start">cron start</a></li>
+<li><a href="#job-killall-job-restart-job-kill">job killall, job restart, job kill</a></li>
+</ul></li>
+<li><a href="#technical-note-about-syntax">Technical Note About Syntax</a></li>
+<li><a href="#caveats">Caveats</a>
+
+<ul>
+<li><a href="#failovers">Failovers</a></li>
+<li><a href="#collision-policy-is-best-effort">Collision policy is best-effort</a></li>
+<li><a href="#timezone-configuration">Timezone Configuration</a></li>
+</ul></li>
+</ul>
+
+<h2 id="overview">Overview</h2>
+
+<p>A job is identified as a cron job by the presence of a
+<code>cron_schedule</code> attribute containing a cron-style schedule in the
+<a href="configuration-reference.md#job-objects"><code>Job</code></a> object. Examples of cron schedules
+include &ldquo;every 5 minutes&rdquo; (<code>*/5 * * * *</code>), &ldquo;Fridays at 17:00&rdquo; (<code>* 17 * * FRI</code>), and
+&ldquo;the 1st and 15th day of the month at 03:00&rdquo; (<code>0 3 1,15 *</code>).</p>
+
+<p>Example (available in the <a href="/documentation/latest/vagrant/">Vagrant environment</a>):</p>
+<pre class="highlight text">$ cat /vagrant/examples/job/cron_hello_world.aurora
+# cron_hello_world.aurora
+# A cron job that runs every 5 minutes.
+jobs = [
+  Job(
+    cluster = &#39;devcluster&#39;,
+    role = &#39;www-data&#39;,
+    environment = &#39;test&#39;,
+    name = &#39;cron_hello_world&#39;,
+    cron_schedule = &#39;*/5 * * * *&#39;,
+    task = SimpleTask(
+      &#39;cron_hello_world&#39;,
+      &#39;echo &quot;Hello world from cron, the time is now $(date --rfc-822)&quot;&#39;),
+  ),
+]
+</pre>
+<h2 id="collision-policies">Collision Policies</h2>
+
+<p>The <code>cron_collision_policy</code> field specifies the scheduler&rsquo;s behavior when a new cron job is
+triggered while an older run hasn&rsquo;t finished. The scheduler has two policies available,
+<a href="#kill_existing">KILL_EXISTING</a> and <a href="#cancel_new">CANCEL_NEW</a>.</p>
+
+<h3 id="kill_existing">KILL_EXISTING</h3>
+
+<p>The default policy - on a collision the old instances are killed and a instances with the current
+configuration are started.</p>
+
+<h3 id="cancel_new">CANCEL_NEW</h3>
+
+<p>On a collision the new run is cancelled.</p>
+
+<p>Note that the use of this flag is likely a code smell - interrupted cron jobs should be able
+to recover their progress on a subsequent invocation, otherwise they risk having their work queue
+grow faster than they can process it.</p>
+
+<h2 id="failure-recovery">Failure recovery</h2>
+
+<p>Unlike with services, which aurora will always re-execute regardless of exit status, instances of
+cron jobs retry according to the <code>max_task_failures</code> attribute of the
+<a href="configuration-reference.md#task-objects">Task</a> object. To get &ldquo;run-until-failure&rdquo; semantics,
+set <code>max_task_failures</code> to <code>-1</code>.</p>
+
+<h2 id="interacting-with-cron-jobs-via-the-aurora-cli">Interacting with cron jobs via the Aurora CLI</h2>
+
+<p>Most interaction with cron jobs takes place using the <code>cron</code> subcommand. See <code>aurora2 help cron</code>
+for up-to-date usage instructions.</p>
+
+<h3 id="cron-schedule">cron schedule</h3>
+
+<p>Schedules a new cron job on the Aurora cluster for later runs or replaces the existing cron template
+with a new one. Only future runs will be affected, any existing active tasks are left intact.</p>
+<pre class="highlight text">$ aurora2 cron schedule devcluster/www-data/test/cron_hello_world /vagrant/examples/jobs/cron_hello_world.aurora
+</pre>
+<h3 id="cron-deschedule">cron deschedule</h3>
+
+<p>Deschedules a cron job, preventing future runs but allowing current runs to complete.</p>
+<pre class="highlight text">$ aurora2 cron deschedule devcluster/www-data/test/cron_hello_world
+</pre>
+<h3 id="cron-start">cron start</h3>
+
+<p>Start a cron job immediately, outside of its normal cron schedule.</p>
+<pre class="highlight text">$ aurora2 cron start devcluster/www-data/test/cron_hello_world
+</pre>
+<h3 id="job-killall,-job-restart,-job-kill">job killall, job restart, job kill</h3>
+
+<p>Cron jobs create instances running on the cluster that you can interact with like normal Aurora
+tasks with <code>job kill</code> and <code>job restart</code>.</p>
+
+<h2 id="technical-note-about-syntax">Technical Note About Syntax</h2>
+
+<p><code>cron_schedule</code> uses a restricted subset of BSD crontab syntax. While the
+execution engine currently uses Quartz, the schedule parsing is custom, a subset of FreeBSD
+<a href="http://www.freebsd.org/cgi/man.cgi?crontab(5)">crontab(5)</a> syntax. See
+<a href="https://github.com/apache/incubator-aurora/blob/master/src/main/java/org/apache/aurora/scheduler/cron/CrontabEntry.java#L106-L124">the source</a>
+for details.</p>
+
+<h2 id="caveats">Caveats</h2>
+
+<h3 id="failovers">Failovers</h3>
+
+<p>No failover recovery. Aurora does not record the latest minute it fired
+triggers for across failovers. Therefore it&rsquo;s possible to miss triggers
+on failover. Note that this behavior may change in the future.</p>
+
+<p>It&rsquo;s necessary to sync time between schedulers with something like <code>ntpd</code>.
+Clock skew could cause double or missed triggers in the case of a failover.</p>
+
+<h3 id="collision-policy-is-best-effort">Collision policy is best-effort</h3>
+
+<p>Aurora aims to always have <em>at least one copy</em> of a given instance running at a time - it&rsquo;s
+an AP system, meaning it chooses Availability and Partition Tolerance at the expense of
+Consistency.</p>
+
+<p>If your collision policy was <code>CANCEL_NEW</code> and a task has terminated but
+Aurora has not noticed this Aurora will go ahead and create your new
+task.</p>
+
+<p>If your collision policy was <code>KILL_EXISTING</code> and a task was marked <code>LOST</code>
+but not yet GCed Aurora will go ahead and create your new task without
+attempting to kill the old one (outside the GC interval).</p>
+
+<h3 id="timezone-configuration">Timezone Configuration</h3>
+
+<p>Cron timezone is configured indepdendently of JVM timezone with the <code>-cron_timezone</code> flag and
+defaults to UTC.</p>
+
+  		</div>
+  	  </div>
+	  
+      	<div class="container-fluid section-footer buffer">
+      <div class="container">
+        <div class="row">
+		  <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+		  <ul>
+		    <li><a href="/downloads/">Downloads</a></li>
+            <li><a href="/community/">Mailing Lists</a></li>
+			<li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+			<li><a href="/documentation/latest/contributing/">How To Contribute</a></li>     
+		  </ul>
+	      </div>
+		  <div class="col-md-2"><h3>The ASF</h3>
+          <ul>
+            <li><a href="http://www.apache.org/licenses/">License</a></li>
+            <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>  
+            <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+            <li><a href="http://www.apache.org/security/">Security</a></li>
+          </ul>
+		  </div>
+		  <div class="col-md-6">
+		    <p class="disclaimer">Apache Aurora is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.</p>
+			<p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+        </div>
+      </div>
+    </div>
+	</body>
+</html>
\ No newline at end of file


Mime
View raw message