aurora-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jfarr...@apache.org
Subject svn commit: r1590066 [1/4] - in /incubator/aurora/site: ./ publish/documentation/latest/ publish/documentation/latest/client-commands/ publish/documentation/latest/clientcommands/ publish/documentation/latest/configuration-reference/ publish/documentat...
Date Fri, 25 Apr 2014 15:59:28 GMT
Author: jfarrell
Date: Fri Apr 25 15:59:27 2014
New Revision: 1590066

URL: http://svn.apache.org/r1590066
Log:
Updating Aurora site with new documentation

Added:
    incubator/aurora/site/publish/documentation/latest/client-commands/
    incubator/aurora/site/publish/documentation/latest/client-commands/index.html
    incubator/aurora/site/publish/documentation/latest/configuration-reference/
    incubator/aurora/site/publish/documentation/latest/configuration-reference/index.html
    incubator/aurora/site/publish/documentation/latest/configuration-tutorial/
    incubator/aurora/site/publish/documentation/latest/configuration-tutorial/index.html
    incubator/aurora/site/publish/documentation/latest/resource-isolation/
    incubator/aurora/site/publish/documentation/latest/resource-isolation/index.html
    incubator/aurora/site/publish/documentation/latest/user-guide/
    incubator/aurora/site/publish/documentation/latest/user-guide/index.html
    incubator/aurora/site/source/documentation/latest/client-commands.md
    incubator/aurora/site/source/documentation/latest/configuration-reference.md
    incubator/aurora/site/source/documentation/latest/configuration-tutorial.md
    incubator/aurora/site/source/documentation/latest/resource-isolation.md
    incubator/aurora/site/source/documentation/latest/user-guide.md
Removed:
    incubator/aurora/site/publish/documentation/latest/clientcommands/index.html
    incubator/aurora/site/publish/documentation/latest/configurationreference/index.html
    incubator/aurora/site/publish/documentation/latest/configurationtutorial/index.html
    incubator/aurora/site/publish/documentation/latest/resourceisolation/index.html
    incubator/aurora/site/publish/documentation/latest/userguide/index.html
    incubator/aurora/site/source/documentation/latest/clientcommands.md
    incubator/aurora/site/source/documentation/latest/configurationreference.md
    incubator/aurora/site/source/documentation/latest/configurationtutorial.md
    incubator/aurora/site/source/documentation/latest/resourceisolation.md
    incubator/aurora/site/source/documentation/latest/userguide.md
Modified:
    incubator/aurora/site/Gemfile
    incubator/aurora/site/Gemfile.lock
    incubator/aurora/site/publish/documentation/latest/deploying-aurora-scheduler/index.html
    incubator/aurora/site/publish/documentation/latest/hooks/index.html
    incubator/aurora/site/publish/documentation/latest/index.html
    incubator/aurora/site/publish/documentation/latest/tutorial/index.html
    incubator/aurora/site/publish/documentation/latest/vagrant/index.html
    incubator/aurora/site/source/documentation/latest.html.md
    incubator/aurora/site/source/documentation/latest/deploying-aurora-scheduler.md
    incubator/aurora/site/source/documentation/latest/hooks.md
    incubator/aurora/site/source/documentation/latest/tutorial.md
    incubator/aurora/site/source/documentation/latest/vagrant.md

Modified: incubator/aurora/site/Gemfile
URL: http://svn.apache.org/viewvc/incubator/aurora/site/Gemfile?rev=1590066&r1=1590065&r2=1590066&view=diff
==============================================================================
--- incubator/aurora/site/Gemfile (original)
+++ incubator/aurora/site/Gemfile Fri Apr 25 15:59:27 2014
@@ -4,4 +4,6 @@ gem 'middleman', '3.2.0'
 gem 'middleman-livereload', '3.1.0'
 gem 'middleman-syntax', '1.2.1'
 
-gem 'redcarpet', github: 'vmg/redcarpet'
\ No newline at end of file
+gem 'redcarpet', github: 'vmg/redcarpet'
+ 
+gem 'rake', '10.3.1'

Modified: incubator/aurora/site/Gemfile.lock
URL: http://svn.apache.org/viewvc/incubator/aurora/site/Gemfile.lock?rev=1590066&r1=1590065&r2=1590066&view=diff
==============================================================================
--- incubator/aurora/site/Gemfile.lock (original)
+++ incubator/aurora/site/Gemfile.lock Fri Apr 25 15:59:27 2014
@@ -75,6 +75,7 @@ GEM
       rack
     rack-test (0.6.2)
       rack (>= 1.0)
+    rake (10.3.1)
     rb-fsevent (0.9.3)
     rb-inotify (0.9.2)
       ffi (>= 0.5.0)
@@ -106,4 +107,5 @@ DEPENDENCIES
   middleman (= 3.2.0)
   middleman-livereload (= 3.1.0)
   middleman-syntax (= 1.2.1)
+  rake (= 10.3.1)
   redcarpet!

Added: incubator/aurora/site/publish/documentation/latest/client-commands/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/client-commands/index.html?rev=1590066&view=auto
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/client-commands/index.html (added)
+++ incubator/aurora/site/publish/documentation/latest/client-commands/index.html Fri Apr 25 15:59:27 2014
@@ -0,0 +1,587 @@
+<html>
+    <head>
+        <meta charset="utf-8">
+        <title>Apache Aurora</title>
+		    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+		    <meta name="description" content="">
+		    <meta name="author" content="">
+
+		    <link href="/assets/css/bootstrap.css" rel="stylesheet">
+		    <link href="/assets/css/bootstrap-responsive.min.css" rel="stylesheet">
+		    <link href="/assets/css/main.css" rel="stylesheet">
+				
+		    <!-- JS -->
+		    <script type="text/javascript" src="/assets/js/jquery-1.10.1.min.js"></script>
+		    <script type="text/javascript" src="/assets/js/bootstrap-dropdown.js"></script>
+		
+				<!-- Analytics -->
+				<script type="text/javascript">
+					  var _gaq = _gaq || [];
+					  _gaq.push(['_setAccount', 'UA-45879646-1']);
+					  _gaq.push(['_setDomainName', 'apache.org']);
+					  _gaq.push(['_trackPageview']);
+
+					  (function() {
+					    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+					    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+					    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+					  })();
+				</script>
+	</head>
+    <body>	
+      <div class="navbar navbar-static-top">
+  <div class="navbar-inner">
+    <div class="container">
+	    <a href="/" class="logo"><img src="/assets/img/aurora_logo.png" alt="Apache Aurora logo" /></a>
+      <ul class="nav">
+				<li><a href="/documentation/latest/">Documentation</a></li>
+        <li><a href="/downloads/">Download</a></li>
+        <li><a href="/community">Community</a></li>
+      </ul>
+    </div>
+  </div>
+</div>
+
+<div class="container">
+<!-- magical breadcrumbs -->
+<ul class="breadcrumb">
+  <li>
+    <div class="dropdown">
+      <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a>
+      <ul class="dropdown-menu" role="menu">
+        <li><a href="http://www.apache.org">Apache Homepage</a></li>
+        <li><a href="http://www.apache.org/licenses/">Apache License</a></li>
+        <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>  
+        <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+        <li><a href="http://www.apache.org/security/">Security</a></li>
+      </ul>
+    </div>
+  </li>
+  <li><span class="divider">&bull;</span></li>
+  <li><a href="http://incubator.apache.org">Apache Incubator</a></li>
+  <li><span class="divider">&bull;</span></li>
+  <li><a href="http://aurora.incubator.apache.org">Apache Aurora</a></li>
+</ul>
+<!-- /breadcrumb -->
+	
+      <div class="container">
+        <h1 id="aurora-client-commands">Aurora Client Commands</h1>
+
+<p>The most up-to-date reference is in the client itself: use the
+<code>aurora help</code> subcommand (for example, <code>aurora help</code> or
+<code>aurora help create</code>) to find the latest information on parameters and
+functionality. Note that <code>aurora help open</code> does not work, due to underlying issues with
+reflection.</p>
+
+<ul>
+<li><a href="#introduction">Introduction</a></li>
+<li><a href="#cluster-configuration">Cluster Configuration</a></li>
+<li><a href="#job-keys">Job Keys</a></li>
+<li><a href="#modifying-aurora-client-commands">Modifying Aurora Client Commands</a></li>
+<li><a href="#regular-jobs">Regular Jobs</a>
+
+<ul>
+<li><a href="#creating-and-running-a-job">Creating and Running a Job</a></li>
+<li><a href="#running-a-command-on-a-running-job">Running a Command On a Running Job</a></li>
+<li><a href="#killing-a-job">Killing a Job</a></li>
+<li><a href="#updating-a-job">Updating a Job</a></li>
+<li><a href="#renaming-a-job">Renaming a Job</a></li>
+<li><a href="#restarting-jobs">Restarting Jobs</a></li>
+</ul></li>
+<li><a href="#cron-jobs">Cron Jobs</a></li>
+<li><a href="#comparing-jobs">Comparing Jobs</a></li>
+<li><a href="#viewingexamining-jobs">Viewing/Examining Jobs</a>
+
+<ul>
+<li><a href="#listing-jobs">Listing Jobs</a></li>
+<li><a href="#inspecting-a-job">Inspecting a Job</a></li>
+<li><a href="#versions">Versions</a></li>
+<li><a href="#checking-your-quota">Checking Your Quota</a></li>
+<li><a href="#finding-a-job-on-web-ui">Finding a Job on Web UI</a></li>
+<li><a href="#getting-job-status">Getting Job Status</a></li>
+<li><a href="#opening-the-web-ui">Opening the Web UI</a></li>
+<li><a href="#sshing-to-a-specific-task-machine">SSHing to a Specific Task Machine</a></li>
+<li><a href="#templating-command-arguments">Templating Command Arguments</a></li>
+</ul></li>
+</ul>
+
+<h2 id="introduction">Introduction</h2>
+
+<p>Once you have written an <code>.aurora</code> configuration file that describes
+your Job and its parameters and functionality, you interact with Aurora
+using Aurora Client commands. This document describes all of these commands
+and how and when to use them. All Aurora Client commands start with
+<code>aurora</code>, followed by the name of the specific command and its
+arguments.</p>
+
+<p><em>Job keys</em> are a very common argument to Aurora commands, as well as the
+gateway to useful information about a Job. Before using Aurora, you
+should read the next section which describes them in detail. The section
+after that briefly describes how you can modify the behavior of certain
+Aurora Client commands, linking to a detailed document about how to do
+that.</p>
+
+<p>This is followed by the Regular Jobs section, which describes the basic
+Client commands for creating, running, and manipulating Aurora Jobs.
+After that are sections on Comparing Jobs and Viewing/Examining Jobs. In
+other words, various commands for getting information and metadata about
+Aurora Jobs.</p>
+
+<h2 id="cluster-configuration">Cluster Configuration</h2>
+
+<p>The client must be able to find a configuration file that speciies available clusters. This file
+declares shorthand names for clusters, which are in turn referenced by job configuration files
+and client commands.</p>
+
+<p>The client will load at most two configuration files, making both of their defined clusters
+available. The first is intended to be a system-installed cluster, using the path specified in
+the environment variable <code>AURORA_CONFIG_ROOT</code>, defaulting to <code>/etc/aurora/clusters.json</code> if the
+environment variable is not set. The second is a user-installed file, located at
+<code>~/.aurora/clusters.json</code>.</p>
+
+<p>A cluster configuration is formatted as JSON.  The simplest cluster configuration is one that
+communicates with a single (non-leader-elected) scheduler.  For example:</p>
+<pre class="highlight javascript"><span class="p">[{</span>
+  <span class="s2">&quot;name&quot;</span><span class="err">:</span> <span class="s2">&quot;example&quot;</span><span class="p">,</span>
+  <span class="s2">&quot;scheduler_uri&quot;</span><span class="err">:</span> <span class="s2">&quot;localhost:55555&quot;</span><span class="p">,</span>
+<span class="p">}]</span>
+</pre>
+<p>A configuration for a leader-elected scheduler would contain something like:</p>
+<pre class="highlight javascript"><span class="p">[{</span>
+  <span class="s2">&quot;name&quot;</span><span class="err">:</span> <span class="s2">&quot;example&quot;</span><span class="p">,</span>
+  <span class="s2">&quot;zk&quot;</span><span class="err">:</span> <span class="s2">&quot;192.168.33.7&quot;</span><span class="p">,</span>
+  <span class="s2">&quot;scheduler_zk_path&quot;</span><span class="err">:</span> <span class="s2">&quot;/aurora/scheduler&quot;</span>
+<span class="p">}]</span>
+</pre>
+<h2 id="job-keys">Job Keys</h2>
+
+<p>A job key is a unique system-wide identifier for an Aurora-managed
+Job, for example <code>cluster1/web-team/test/experiment204</code>. It is a 4-tuple
+consisting of, in order, <em>cluster</em>, <em>role</em>, <em>environment</em>, and
+<em>jobname</em>, separated by /s. Cluster is the name of an Aurora
+cluster. Role is the Unix service account under which the Job
+runs. Environment is a namespace component like <code>devel</code>, <code>test</code>,
+<code>prod</code>, or <code>stagingN.</code> Jobname is the Job&rsquo;s name.</p>
+
+<p>The combination of all four values uniquely specifies the Job. If any
+one value is different from that of another job key, the two job keys
+refer to different Jobs. For example, job key
+<code>cluster1/tyg/prod/workhorse</code> is different from
+<code>cluster1/tyg/prod/workcamel</code> is different from
+<code>cluster2/tyg/prod/workhorse</code> is different from
+<code>cluster2/foo/prod/workhorse</code> is different from
+<code>cluster1/tyg/test/workhorse.</code></p>
+
+<p>Role names are user accounts existing on the slave machines. If you don&rsquo;t know what accounts
+are available, contact your sysadmin.</p>
+
+<p>Environment names are namespaces; you can count on <code>prod</code>, <code>devel</code> and <code>test</code> existing.</p>
+
+<h2 id="modifying-aurora-client-commands">Modifying Aurora Client Commands</h2>
+
+<p>For certain Aurora Client commands, you can define hook methods that run
+either before or after an action that takes place during the command&rsquo;s
+execution, as well as based on whether the action finished successfully or failed
+during execution. Basically, a hook is code that lets you extend the
+command&rsquo;s actions. The hook executes on the client side, specifically on
+the machine executing Aurora commands.</p>
+
+<p>Hooks can be associated with these Aurora Client commands.</p>
+
+<ul>
+<li><code>cancel_update</code></li>
+<li><code>create</code></li>
+<li><code>kill</code></li>
+<li><code>restart</code></li>
+<li><code>update</code></li>
+</ul>
+
+<p>The process for writing and activating them is complex enough
+that we explain it in a devoted document, <a href="/documentation/latest/hooks/">Hooks for Aurora Client API</a>.</p>
+
+<h2 id="regular-jobs">Regular Jobs</h2>
+
+<p>This section covers Aurora commands related to running, killing,
+renaming, updating, and restarting a basic Aurora Job.</p>
+
+<h3 id="creating-and-running-a-job">Creating and Running a Job</h3>
+<pre class="highlight text">aurora create &lt;job key&gt; &lt;configuration file&gt;
+</pre>
+<p>Creates and then runs a Job with the specified job key based on a <code>.aurora</code> configuration file.
+The configuration file may also contain and activate hook definitions.</p>
+
+<p><code>create</code> can take four named parameters:</p>
+
+<ul>
+<li><code>-E NAME=VALUE</code> Bind a Thermos mustache variable name to a
+value. Multiple flags specify multiple values. Defaults to <code>[]</code>.</li>
+<li><code>-o, --open_browser</code> Open a browser window to the scheduler UI Job
+page after a job changing operation happens. When <code>False</code>, the Job
+URL prints on the console and the user has to copy/paste it
+manually. Defaults to <code>False</code>. Does not work when running in Vagrant.</li>
+<li><code>-j, --json</code> If specified, configuration argument is read as a
+string in JSON format. Defaults to False.</li>
+<li><code>--wait_until=STATE</code> Block the client until all the Tasks have
+transitioned into the requested state. Possible values are: <code>PENDING</code>,
+<code>RUNNING</code>, <code>FINISHED</code>. Default: <code>PENDING</code></li>
+</ul>
+
+<h3 id="running-a-command-on-a-running-job">Running a Command On a Running Job</h3>
+<pre class="highlight text">aurora run &lt;job_key&gt; &lt;cmd&gt;
+</pre>
+<p>Runs a shell command on all machines currently hosting shards of a
+single Job.</p>
+
+<p><code>run</code> supports the same command line wildcards used to populate a Job&rsquo;s
+commands; i.e. anything in the <code>{{mesos.*}}</code> and <code>{{thermos.*}}</code>
+namespaces.</p>
+
+<p><code>run</code> can take three named parameters:</p>
+
+<ul>
+<li><code>-t NUM_THREADS</code>, <code>--threads=NUM_THREADS</code>The number of threads to
+use, defaulting to <code>1</code>.</li>
+<li><code>--user=SSH_USER</code> ssh as this user instead of the given role value.
+Defaults to None.</li>
+<li><code>-e, --executor_sandbox</code>  Run the command in the executor sandbox
+instead of the Task sandbox. Defaults to False.</li>
+</ul>
+
+<h3 id="killing-a-job">Killing a Job</h3>
+<pre class="highlight text">aurora kill &lt;job key&gt; &lt;configuration file&gt;
+</pre>
+<p>Kills all Tasks associated with the specified Job, blocking until all
+are terminated. Defaults to killing all shards in the Job.</p>
+
+<p>The <code>&lt;configuration file&gt;</code> argument for <code>kill</code> is optional. Use it only
+if it contains hook definitions and activations that affect the
+kill command.</p>
+
+<p><code>kill</code> can take two named parameters:</p>
+
+<ul>
+<li><code>-o, --open_browser</code> Open a browser window to the scheduler UI Job
+page after a job changing operation happens. When <code>False</code>, the Job
+URL prints on the console and the user has to copy/paste it
+manually. Defaults to <code>False</code>. Does not work when running in Vagrant.</li>
+<li><code>--shards=SHARDS</code> A list of shard ids to act on. Can either be a
+comma-separated list (e.g. 0,1,2) or a range (e.g. 0-2) or  any
+combination of the two (e.g. 0-2,5,7-9). Defaults to acting on all
+shards.</li>
+</ul>
+
+<h3 id="updating-a-job">Updating a Job</h3>
+<pre class="highlight text">aurora update [--shards=ids] &lt;job key&gt; &lt;configuration file&gt;
+aurora cancel_update &lt;job key&gt; &lt;configuration file&gt;
+</pre>
+<p>Given a running job, does a rolling update to reflect a new
+configuration version. Only updates Tasks in the Job with a changed
+configuration. You can further restrict the operated on Tasks by
+using <code>--shards</code> and specifying a comma-separated list of job shard ids.</p>
+
+<p>You may want to run <code>aurora diff</code> beforehand to validate which Tasks
+have different configurations.</p>
+
+<p>Updating jobs are locked to be sure the update finishes without
+disruption. If the update abnormally terminates, the lock may stay
+around and cause failure of subsequent update attempts.
+ <code>aurora cancel_update</code>unlocks the Job specified by
+its <code>job_key</code> argument. Be sure you don&rsquo;t issue <code>cancel_update</code> when
+another user is working with the specified Job.</p>
+
+<p>The <code>&lt;configuration file&gt;</code> argument for <code>cancel_update</code> is optional. Use
+it only if it contains hook definitions and activations that affect the
+<code>cancel_update</code> command. The <code>&lt;configuration file&gt;</code> argument for
+<code>update</code> is required, but in addition to a new configuration it can be
+used to define and activate hooks for <code>update</code>.</p>
+
+<p><code>update</code> can take four named parameters:</p>
+
+<ul>
+<li><code>--shards=SHARDS</code> A list of shard ids to update. Can either be a
+comma-separated list (e.g. 0,1,2) or a range (e.g. 0-2) or  any
+combination of the two (e.g. 0-2,5,7-9). If not  set, all shards are
+acted on. Defaults to None.</li>
+<li><code>-E NAME=VALUE</code> Binds a Thermos mustache variable name to a value.
+Use multiple flags to specify multiple values. Defaults to <code>[]</code>.</li>
+<li><code>-j, --json</code> If specified, configuration is read in JSON format.
+Defaults to <code>False</code>.</li>
+<li><code>--updater_health_check_interval_seconds=HEALTH_CHECK_INTERVAL_SECONDS</code>
+Time interval between subsequent shard status checks. Defaults to <code>3</code>.</li>
+</ul>
+
+<h3 id="renaming-a-job">Renaming a Job</h3>
+
+<p>Renaming is a tricky operation as downstream clients must be informed of
+the new name. A conservative approach
+to renaming suitable for production services is:</p>
+
+<ol>
+<li> Modify the Aurora configuration file to change the role,
+environment, and/or name as appropriate to the standardized naming
+scheme.</li>
+<li><p>Check that only these naming components have changed
+with <code>aurora diff</code>.</p>
+<pre class="highlight text">aurora diff &lt;job_key&gt; &lt;job_configuration&gt;
+</pre></li>
+<li><p>Create the (identical) job at the new key. You may need to request a
+temporary quota increase.</p>
+<pre class="highlight text">aurora create &lt;new_job_key&gt; &lt;job_configuration&gt;
+</pre></li>
+<li><p>Migrate all clients over to the new job key. Update all links and
+dashboards. Ensure that both job keys run identical versions of the
+code while in this state.</p></li>
+<li><p>After verifying that all clients have successfully moved over, kill
+the old job.</p>
+<pre class="highlight text">aurora kill &lt;old_job_key&gt;
+</pre></li>
+<li><p>If you received a temporary quota increase, be sure to let the
+powers that be know you no longer need the additional capacity.</p></li>
+</ol>
+
+<h3 id="restarting-jobs">Restarting Jobs</h3>
+
+<p><code>restart</code> restarts all of a job key identified Job&rsquo;s shards:</p>
+<pre class="highlight text">aurora restart &lt;job_key&gt; &lt;configuration file&gt;
+</pre>
+<p>Restarts are controlled on the client side, so aborting
+the <code>restart</code> command halts the restart operation.</p>
+
+<p><code>restart</code> does a rolling restart. You almost always want to do this, but
+not if all shards of a service are misbehaving and are
+completely dysfunctional. To not do a rolling restart, use
+the <code>-shards</code> option described below.</p>
+
+<p><strong>Note</strong>: <code>restart</code> only applies its command line arguments and does not
+use or is affected by <code>update.config</code>. Restarting
+does <strong><em>not</em></strong> involve a configuration change. To update the
+configuration, use <code>update.config</code>.</p>
+
+<p>The <code>&lt;configuration file&gt;</code> argument for restart is optional. Use it only
+if it contains hook definitions and activations that affect the
+<code>restart</code> command.</p>
+
+<p>In addition to the required job key argument, there are eight
+<code>restart</code> specific optional arguments:</p>
+
+<ul>
+<li><code>--updater_health_check_interval_seconds</code>: Defaults to <code>3</code>, the time
+interval between subsequent shard status checks.</li>
+<li><code>--shards=SHARDS</code>: Defaults to None, which restarts all shards.
+Otherwise, only the specified-by-id shards restart. They can be
+comma-separated <code>(0, 8, 9)</code>, a range <code>(3-5)</code> or a
+combination <code>(0, 3-5, 8, 9-11)</code>.</li>
+<li><code>--batch_size</code>: Defaults to <code>1</code>, the number of shards to be started
+in one iteration. So, for example, for value 3, it tries to restart
+the first three shards specified by <code>--shards</code> simultaneously, then
+the next three, and so on.</li>
+<li><code>--max_per_shard_failures=MAX_PER_SHARD_FAILURES</code>: Defaults to <code>0</code>,
+the maximum number of restarts per shard during restart. When
+exceeded, it increments the total failure count.</li>
+<li><code>--max_total_failures=MAX_TOTAL_FAILURES</code>: Defaults to <code>0</code>, the
+maximum total number of shard failures tolerated during restart.</li>
+<li><code>-o, --open_browser</code> Open a browser window to the scheduler UI Job
+page after a job changing operation happens. When <code>False</code>, the Job
+url prints on the console and the user has to copy/paste it
+manually. Defaults to <code>False</code>. Does not work when running in Vagrant.</li>
+<li><code>--restart_threshold</code>: Defaults to <code>60</code>, the maximum number of
+seconds before a shard must move into the <code>RUNNING</code> state before
+it&rsquo;s considered a failure.</li>
+<li><code>--watch_secs</code>: Defaults to <code>30</code>, the minimum number of seconds a
+shard must remain in <code>RUNNING</code> state before considered a success.</li>
+</ul>
+
+<h2 id="cron-jobs">Cron Jobs</h2>
+
+<p>You will see various commands and options relating to cron jobs in
+<code>aurora -help</code> and similar. Ignore them, as they&rsquo;re not yet implemented.
+You might be able to use them without causing an error, but nothing happens
+if you do.</p>
+
+<h2 id="comparing-jobs">Comparing Jobs</h2>
+<pre class="highlight text">aurora diff &lt;job_key&gt; config
+</pre>
+<p>Compares a job configuration against a running job. By default the diff
+is determined using <code>diff</code>, though you may choose an alternate
+ diff program by specifying the <code>DIFF_VIEWER</code> environment variable.</p>
+
+<p>There are two named parameters:</p>
+
+<ul>
+<li><code>-E NAME=VALUE</code> Bind a Thermos mustache variable name to a
+value. Multiple flags may be used to specify multiple values.
+Defaults to <code>[]</code>.</li>
+<li><code>-j, --json</code> Read the configuration argument in JSON format.
+Defaults to <code>False</code>.</li>
+</ul>
+
+<h2 id="viewing/examining-jobs">Viewing/Examining Jobs</h2>
+
+<p>Above we discussed creating, killing, and updating Jobs. Here we discuss
+how to view and examine Jobs.</p>
+
+<h3 id="listing-jobs">Listing Jobs</h3>
+<pre class="highlight text">aurora list_jobs
+Usage: `aurora list_jobs cluster/role
+</pre>
+<p>Lists all Jobs registered with the Aurora scheduler in the named cluster for the named role.</p>
+
+<p>It has two named parameters:</p>
+
+<ul>
+<li><code>--pretty</code>: Displays job information in prettyprinted format.
+Defaults to <code>False</code>.</li>
+<li><code>-c</code>, <code>--show-cron</code>: Shows cron schedule for jobs. Defaults to
+<code>False</code>. Do not use, as it&rsquo;s not yet implemented.</li>
+</ul>
+
+<h3 id="inspecting-a-job">Inspecting a Job</h3>
+<pre class="highlight text">aurora inspect &lt;job_key&gt; config
+</pre>
+<p><code>inspect</code> verifies that its specified job can be parsed from a
+configuration file, and displays the parsed configuration. It has four
+named parameters:</p>
+
+<ul>
+<li><code>--local</code>: Inspect the configuration that the  <code>spawn</code> command would
+create, defaulting to <code>False</code>.</li>
+<li><code>--raw</code>: Shows the raw configuration. Defaults to <code>False</code>.</li>
+<li><code>-j</code>, <code>--json</code>: If specified, configuration is read in JSON format.
+Defaults to <code>False</code>.</li>
+<li><code>-E NAME=VALUE</code>: Bind a Thermos Mustache variable name to a value.
+You can use multiple flags to specify multiple values. Defaults
+to <code>[]</code></li>
+</ul>
+
+<h3 id="versions">Versions</h3>
+<pre class="highlight text">aurora version
+</pre>
+<p>Lists client build information and what Aurora API version it supports.</p>
+
+<h3 id="checking-your-quota">Checking Your Quota</h3>
+<pre class="highlight text">aurora get_quota --cluster=CLUSTER role
+</pre>
+<p>Prints the production quota allocated to the role&rsquo;s value at the given
+cluster.</p>
+
+<h3 id="finding-a-job-on-web-ui">Finding a Job on Web UI</h3>
+
+<p>When you create a job, part of the output response contains a URL that goes
+to the job&rsquo;s scheduler UI page. For example:</p>
+<pre class="highlight text">vagrant@precise64:~$ aurora create example/www-data/prod/hello /vagrant/examples/jobs/hello_world.aurora
+INFO] Creating job hello
+INFO] Response from scheduler: OK (message: 1 new tasks pending for job www-data/prod/hello)
+INFO] Job url: http://precise64:8081/scheduler/www-data/prod/hello
+</pre>
+<p>You can go to the scheduler UI page for this job via <code>http://precise64:8081/scheduler/www-data/prod/hello</code>
+You can go to the overall scheduler UI page by going to the part of that URL that ends at <code>scheduler</code>; <code>http://precise64:8081/scheduler</code></p>
+
+<p>Once you click through to a role page, you see Jobs arranged
+separately by pending jobs, active jobs and finished jobs.
+Jobs are arranged by role, typically a service account for
+production jobs and user accounts for test or development jobs.</p>
+
+<h3 id="getting-job-status">Getting Job Status</h3>
+<pre class="highlight text">aurora status &lt;job_key&gt;
+</pre>
+<p>Returns the status of recent tasks associated with the
+<code>job_key</code> specified Job in its supplied cluster. Typically this includes
+a mix of active tasks (running or assigned) and inactive tasks
+(successful, failed, and lost.)</p>
+
+<h3 id="opening-the-web-ui">Opening the Web UI</h3>
+
+<p>Use the Job&rsquo;s web UI scheduler URL or the <code>aurora status</code> command to find out on which
+machines individual tasks are scheduled. You can open the web UI via the
+<code>open</code> command line command if invoked from your machine:</p>
+<pre class="highlight text">aurora open [&lt;cluster&gt;[/&lt;role&gt;[/&lt;env&gt;/&lt;job_name&gt;]]]
+</pre>
+<p>If only the cluster is specified, it goes directly to that cluster&rsquo;s
+scheduler main page. If the role is specified, it goes to the top-level
+role page. If the full job key is specified, it goes directly to the job
+page where you can inspect individual tasks.</p>
+
+<h3 id="sshing-to-a-specific-task-machine">SSHing to a Specific Task Machine</h3>
+<pre class="highlight text">aurora ssh &lt;job_key&gt; &lt;shard number&gt;
+</pre>
+<p>You can have the Aurora client ssh directly to the machine that has been
+assigned a particular Job/shard number. This may be useful for quickly
+diagnosing issues such as performance issues or abnormal behavior on a
+particular machine.</p>
+
+<p>It can take three named parameters:</p>
+
+<ul>
+<li><code>-e</code>, <code>--executor_sandbox</code>:  Run <code>ssh</code> in the executor sandbox
+instead of the  task sandbox. Defaults to <code>False</code>.</li>
+<li><code>--user=SSH_USER</code>: <code>ssh</code> as the given user instead of as the role in
+the <code>job_key</code> argument. Defaults to none.</li>
+<li><code>-L PORT:NAME</code>: Add tunnel from local port <code>PORT</code> to the remote
+named port  <code>NAME</code>. Defaults to <code>[]</code>.</li>
+</ul>
+
+<h3 id="templating-command-arguments">Templating Command Arguments</h3>
+<pre class="highlight text">aurora run [-e] [-t THREADS] &lt;job_key&gt; -- &lt;&lt;command-line&gt;&gt;
+</pre>
+<p>Given a job specification, run the supplied command on all hosts and
+return the output. You may use the standard Mustache templating rules:</p>
+
+<ul>
+<li><code>{{thermos.ports[name]}}</code> substitutes the specific named port of the
+task assigned to this machine</li>
+<li><code>{{mesos.instance}}</code> substitutes the shard id of the job&rsquo;s task
+assigned to this machine</li>
+<li><code>{{thermos.task_id}}</code> substitutes the task id of the job&rsquo;s task
+assigned to this machine</li>
+</ul>
+
+<p>For example, the following type of pattern can be a powerful diagnostic
+tool:</p>
+<pre class="highlight text">aurora run -t5 cluster1/tyg/devel/seizure -- \
+  &#39;curl -s -m1 localhost:{{thermos.ports[http]}}/vars | grep uptime&#39;
+</pre>
+<p>By default, the command runs in the Task&rsquo;s sandbox. The <code>-e</code> option can
+run the command in the executor&rsquo;s sandbox. This is mostly useful for
+Aurora administrators.</p>
+
+<p>You can parallelize the runs by using the <code>-t</code> option.</p>
+
+	  </div>
+      <div class="container">
+    <hr>
+    <footer class="footer">
+        <div class="row-fluid">
+            <div class="span2 text-left">
+                <h3>Links</h3>
+                <ul class="unstyled">
+                    <li><a href="/downloads/">Downloads</a></li>
+                    <li><a href="/developers/">Developers</a></li>                    
+                </ul>
+            </div>
+            <div class="span3 text-left">
+                <h3>Community</h3>
+                <ul class="unstyled">
+                    <li><a href="/community/">Mailing Lists</a></li>
+                    <li><a href="http://issues.apache.org/jira/browse/aurora">Issue Tracking</a></li>
+                    <li><a href="/docs/howtocontribute/">How To Contribute</a></li>
+                </ul>
+            </div>
+            <div class="span7 text-left">
+            	<h3>Apache Software Foundation</h3>
+
+							<div class="span8">
+                Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. Apache, Apache Thrift, and the Apache feather logo are trademarks of The Apache Software Foundation. Currently part of the <a href="http://incubator.apache.org">Apache Incubator</a>.
+							</div>
+							<div class=" pull-right">
+								<a href="http://incubator.apache.org" class="logo"><img src="/assets/img/apache_incubator_logo.png" alt="Apache Incubator" class="pull-right"/></a>
+							</div>
+            </div>
+
+        </div>
+
+    </footer>
+</div>
+
+	</body>
+</html>
+

Added: incubator/aurora/site/publish/documentation/latest/configuration-reference/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/configuration-reference/index.html?rev=1590066&view=auto
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/configuration-reference/index.html (added)
+++ incubator/aurora/site/publish/documentation/latest/configuration-reference/index.html Fri Apr 25 15:59:27 2014
@@ -0,0 +1,870 @@
+<html>
+    <head>
+        <meta charset="utf-8">
+        <title>Apache Aurora</title>
+		    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+		    <meta name="description" content="">
+		    <meta name="author" content="">
+
+		    <link href="/assets/css/bootstrap.css" rel="stylesheet">
+		    <link href="/assets/css/bootstrap-responsive.min.css" rel="stylesheet">
+		    <link href="/assets/css/main.css" rel="stylesheet">
+				
+		    <!-- JS -->
+		    <script type="text/javascript" src="/assets/js/jquery-1.10.1.min.js"></script>
+		    <script type="text/javascript" src="/assets/js/bootstrap-dropdown.js"></script>
+		
+				<!-- Analytics -->
+				<script type="text/javascript">
+					  var _gaq = _gaq || [];
+					  _gaq.push(['_setAccount', 'UA-45879646-1']);
+					  _gaq.push(['_setDomainName', 'apache.org']);
+					  _gaq.push(['_trackPageview']);
+
+					  (function() {
+					    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+					    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+					    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+					  })();
+				</script>
+	</head>
+    <body>	
+      <div class="navbar navbar-static-top">
+  <div class="navbar-inner">
+    <div class="container">
+	    <a href="/" class="logo"><img src="/assets/img/aurora_logo.png" alt="Apache Aurora logo" /></a>
+      <ul class="nav">
+				<li><a href="/documentation/latest/">Documentation</a></li>
+        <li><a href="/downloads/">Download</a></li>
+        <li><a href="/community">Community</a></li>
+      </ul>
+    </div>
+  </div>
+</div>
+
+<div class="container">
+<!-- magical breadcrumbs -->
+<ul class="breadcrumb">
+  <li>
+    <div class="dropdown">
+      <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a>
+      <ul class="dropdown-menu" role="menu">
+        <li><a href="http://www.apache.org">Apache Homepage</a></li>
+        <li><a href="http://www.apache.org/licenses/">Apache License</a></li>
+        <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>  
+        <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+        <li><a href="http://www.apache.org/security/">Security</a></li>
+      </ul>
+    </div>
+  </li>
+  <li><span class="divider">&bull;</span></li>
+  <li><a href="http://incubator.apache.org">Apache Incubator</a></li>
+  <li><span class="divider">&bull;</span></li>
+  <li><a href="http://aurora.incubator.apache.org">Apache Aurora</a></li>
+</ul>
+<!-- /breadcrumb -->
+	
+      <div class="container">
+        <h1 id="aurora-+-thermos-configuration-reference">Aurora + Thermos Configuration Reference</h1>
+
+<ul>
+<li><a href="#aurora--thermos-configuration-reference">Aurora + Thermos Configuration Reference</a></li>
+<li><a href="#introduction">Introduction</a></li>
+<li><a href="#process-schema">Process Schema</a>
+
+<ul>
+<li><a href="#process-objects">Process Objects</a>
+
+<ul>
+<li><a href="#name">name</a></li>
+<li><a href="#cmdline">cmdline</a></li>
+<li><a href="#max_failures">max_failures</a></li>
+<li><a href="#daemon">daemon</a></li>
+<li><a href="#ephemeral">ephemeral</a></li>
+<li><a href="#min_duration">min_duration</a></li>
+<li><a href="#final">final</a></li>
+</ul></li>
+</ul></li>
+<li><a href="#task-schema">Task Schema</a>
+
+<ul>
+<li><a href="#task-object">Task Object</a>
+
+<ul>
+<li><a href="#name-1">name</a></li>
+<li><a href="#processes">processes</a></li>
+<li><a href="#constraints">constraints</a></li>
+<li><a href="#resources">resources</a></li>
+<li><a href="#max_failures-1">max_failures</a></li>
+<li><a href="#max_concurrency">max_concurrency</a></li>
+<li><a href="#finalization_wait">finalization_wait</a></li>
+</ul></li>
+<li><a href="#constraint-object">Constraint Object</a></li>
+<li><a href="#resource-object">Resource Object</a></li>
+</ul></li>
+<li><a href="#job-schema">Job Schema</a>
+
+<ul>
+<li><a href="#job-objects">Job Objects</a></li>
+<li><a href="#services">Services</a></li>
+<li><a href="#updateconfig-objects">UpdateConfig Objects</a></li>
+<li><a href="#healthcheckconfig-objects">HealthCheckConfig Objects</a></li>
+</ul></li>
+<li><a href="#specifying-scheduling-constraints">Specifying Scheduling Constraints</a></li>
+<li><a href="#template-namespaces">Template Namespaces</a>
+
+<ul>
+<li><a href="#mesos-namespace">mesos Namespace</a></li>
+<li><a href="#thermos-namespace">thermos Namespace</a></li>
+</ul></li>
+<li><a href="#basic-examples">Basic Examples</a>
+
+<ul>
+<li><a href="#hello_worldaurora">hello_world.aurora</a></li>
+<li><a href="#environment-tailoring">Environment Tailoring</a>
+
+<ul>
+<li><a href="#hello_world_productionizedaurora">hello<em>world</em>productionized.aurora</a></li>
+</ul></li>
+</ul></li>
+</ul>
+
+<h1 id="introduction">Introduction</h1>
+
+<p>Don&rsquo;t know where to start? The Aurora configuration schema is very
+powerful, and configurations can become quite complex for advanced use
+cases.</p>
+
+<p>For examples of simple configurations to get something up and running
+quickly, check out the <a href="/documentation/latest/tutorial/">Tutorial</a>. When you feel comfortable with the basics, move
+on to the <a href="/documentation/latest/configuration-tutorial/">Configuration Tutorial</a> for more in-depth coverage of
+configuration design.</p>
+
+<p>For additional basic configuration examples, see <a href="#BasicExamples">the end of this document</a>.</p>
+
+<h1 id="process-schema">Process Schema</h1>
+
+<p>Process objects consist of required <code>name</code> and <code>cmdline</code> attributes. You can customize Process
+behavior with its optional attributes. Remember, Processes are handled by Thermos.</p>
+
+<h3 id="process-objects">Process Objects</h3>
+
+<table><thead>
+<tr>
+<th><strong>Attribute Name</strong></th>
+<th style="text-align: center"><strong>Type</strong></th>
+<th><strong>Description</strong></th>
+</tr>
+</thead><tbody>
+<tr>
+<td><strong>name</strong></td>
+<td style="text-align: center">String</td>
+<td>Process name (Required)</td>
+</tr>
+<tr>
+<td><strong>cmdline</strong></td>
+<td style="text-align: center">String</td>
+<td>Command line (Required)</td>
+</tr>
+<tr>
+<td><strong>max_failures</strong></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum process failures (Default: 1)</td>
+</tr>
+<tr>
+<td><strong>daemon</strong></td>
+<td style="text-align: center">Boolean</td>
+<td>When True, this is a daemon process. (Default: False)</td>
+</tr>
+<tr>
+<td><strong>ephemeral</strong></td>
+<td style="text-align: center">Boolean</td>
+<td>When True, this is an ephemeral process. (Default: False)</td>
+</tr>
+<tr>
+<td><strong>min_duration</strong></td>
+<td style="text-align: center">Integer</td>
+<td>Minimum duration between process restarts in seconds. (Default: 15)</td>
+</tr>
+<tr>
+<td><strong>final</strong></td>
+<td style="text-align: center">Boolean</td>
+<td>When True, this process is a finalizing one that should run last. (Default: False)</td>
+</tr>
+</tbody></table>
+
+<h4 id="name">name</h4>
+
+<p>The name is any valid UNIX filename string (specifically no
+slashes, NULLs or leading periods). Within a Task object, each Process name
+must be unique.</p>
+
+<h4 id="cmdline">cmdline</h4>
+
+<p>The command line run by the process. The command line is invoked in a bash
+subshell, so can involve fully-blown bash scripts. However, nothing is
+supplied for command-line arguments so <code>$*</code> is unspecified.</p>
+
+<h4 id="max_failures">max_failures</h4>
+
+<p>The maximum number of failures (non-zero exit statuses) this process can
+have before being marked permanently failed and not retried. If a
+process permanently fails, Thermos looks at the failure limit of the task
+containing the process (usually 1) to determine if the task has
+failed as well.</p>
+
+<p>Setting <code>max_failures</code> to 0 makes the process retry
+indefinitely until it achieves a successful (zero) exit status.
+It retries at most once every <code>min_duration</code> seconds to prevent
+an effective denial of service attack on the coordinating Thermos scheduler.</p>
+
+<h4 id="daemon">daemon</h4>
+
+<p>By default, Thermos processes are non-daemon. If <code>daemon</code> is set to True, a
+successful (zero) exit status does not prevent future process runs.
+Instead, the process reinvokes after <code>min_duration</code> seconds.
+However, the maximum failure limit still applies. A combination of
+<code>daemon=True</code> and <code>max_failures=0</code> causes a process to retry
+indefinitely regardless of exit status. This should be avoided
+for very short-lived processes because of the accumulation of
+checkpointed state for each process run. When running in Mesos
+specifically, <code>max_failures</code> is capped at 100.</p>
+
+<h4 id="ephemeral">ephemeral</h4>
+
+<p>By default, Thermos processes are non-ephemeral. If <code>ephemeral</code> is set to
+True, the process&#39; status is not used to determine if its containing task
+has completed. For example, consider a task with a non-ephemeral
+webserver process and an ephemeral logsaver process
+that periodically checkpoints its log files to a centralized data store.
+The task is considered finished once the webserver process has
+completed, regardless of the logsaver&rsquo;s current status.</p>
+
+<h4 id="min_duration">min_duration</h4>
+
+<p>Processes may succeed or fail multiple times during a single task&rsquo;s
+duration. Each of these is called a <em>process run</em>. <code>min_duration</code> is
+the minimum number of seconds the scheduler waits before running the
+same process.</p>
+
+<h4 id="final">final</h4>
+
+<p>Processes can be grouped into two classes: ordinary processes and
+finalizing processes. By default, Thermos processes are ordinary. They
+run as long as the task is considered healthy (i.e., no failure
+limits have been reached.) But once all regular Thermos processes
+finish or the task reaches a certain failure threshold, it
+moves into a &ldquo;finalization&rdquo; stage and runs all finalizing
+processes. These are typically processes necessary for cleaning up the
+task, such as log checkpointers, or perhaps e-mail notifications that
+the task completed.</p>
+
+<p>Finalizing processes may not depend upon ordinary processes or
+vice-versa, however finalizing processes may depend upon other
+finalizing processes and otherwise run as a typical process
+schedule.</p>
+
+<h1 id="task-schema">Task Schema</h1>
+
+<p>Tasks fundamentally consist of a <code>name</code> and a list of Process objects stored as the
+value of the <code>processes</code> attribute. Processes can be further constrained with
+<code>constraints</code>. By default, <code>name</code>&rsquo;s value inherits from the first Process in the
+<code>processes</code> list, so for simple <code>Task</code> objects with one Process, <code>name</code>
+can be omitted. In Mesos, <code>resources</code> is also required.</p>
+
+<h3 id="task-object">Task Object</h3>
+
+<table><thead>
+<tr>
+<th><strong>param</strong></th>
+<th style="text-align: center"><strong>type</strong></th>
+<th><strong>description</strong></th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>name</code></td>
+<td style="text-align: center">String</td>
+<td>Process name (Required) (Default: <code>processes0.name</code>)</td>
+</tr>
+<tr>
+<td><code>processes</code></td>
+<td style="text-align: center">List of <code>Process</code> objects</td>
+<td>List of <code>Process</code> objects bound to this task. (Required)</td>
+</tr>
+<tr>
+<td><code>constraints</code></td>
+<td style="text-align: center">List of <code>Constraint</code> objects</td>
+<td>List of <code>Constraint</code> objects constraining processes.</td>
+</tr>
+<tr>
+<td><code>resources</code></td>
+<td style="text-align: center"><code>Resource</code> object</td>
+<td>Resource footprint. (Required)</td>
+</tr>
+<tr>
+<td><code>max_failures</code></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum process failures before being considered failed (Default: 1)</td>
+</tr>
+<tr>
+<td><code>max_concurrency</code></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum number of concurrent processes (Default: 0, unlimited concurrency.)</td>
+</tr>
+<tr>
+<td><code>finalization_wait</code></td>
+<td style="text-align: center">Integer</td>
+<td>Amount of time allocated for finalizing processes, in seconds. (Default: 30)</td>
+</tr>
+</tbody></table>
+
+<h4 id="name">name</h4>
+
+<p><code>name</code> is a string denoting the name of this task. It defaults to the name of the first Process in
+the list of Processes associated with the <code>processes</code> attribute.</p>
+
+<h4 id="processes">processes</h4>
+
+<p><code>processes</code> is an unordered list of <code>Process</code> objects. To constrain the order
+in which they run, use <code>constraints</code>.</p>
+
+<h5 id="constraints">constraints</h5>
+
+<p>A list of <code>Constraint</code> objects. Currently it supports only one type,
+the <code>order</code> constraint. <code>order</code> is a list of process names
+that should run in the order given. For example,</p>
+<pre class="highlight text">    process = Process(cmdline = &quot;echo hello {{name}}&quot;)
+    task = Task(name = &quot;echoes&quot;,
+                processes = [process(name = &quot;jim&quot;), process(name = &quot;bob&quot;)],
+                constraints = [Constraint(order = [&quot;jim&quot;, &quot;bob&quot;]))
+</pre>
+<p>Constraints can be supplied ad-hoc and in duplicate. Not all
+Processes need be constrained, however Tasks with cycles are
+rejected by the Thermos scheduler.</p>
+
+<p>Use the <code>order</code> function as shorthand to generate <code>Constraint</code> lists.
+The following:</p>
+<pre class="highlight text">    order(process1, process2)
+</pre>
+<p>is shorthand for</p>
+<pre class="highlight text">    [Constraint(order = [process1.name(), process2.name()])]
+</pre>
+<h4 id="resources">resources</h4>
+
+<p>Takes a <code>Resource</code> object, which specifies the amounts of CPU, memory, and disk space resources
+to allocate to the Task.</p>
+
+<h4 id="max_failures">max_failures</h4>
+
+<p><code>max_failures</code> is the number of times processes that are part of this
+Task can fail before the entire Task is marked for failure.</p>
+
+<p>For example:</p>
+<pre class="highlight text">    template = Process(max_failures=10)
+    task = Task(
+      name = &quot;fail&quot;,
+      processes = [
+         template(name = &quot;failing&quot;, cmdline = &quot;exit 1&quot;),
+         template(name = &quot;succeeding&quot;, cmdline = &quot;exit 0&quot;)
+      ],
+      max_failures=2)
+</pre>
+<p>The <code>failing</code> Process could fail 10 times before being marked as
+permanently failed, and the <code>succeeding</code> Process would succeed on the
+first run. The task would succeed despite only allowing for two failed
+processes. To be more specific, there would be 10 failed process runs
+yet 1 failed process.</p>
+
+<h4 id="max_concurrency">max_concurrency</h4>
+
+<p>For Tasks with a number of expensive but otherwise independent
+processes, you may want to limit the amount of concurrency
+the Thermos scheduler provides rather than artificially constraining
+it via <code>order</code> constraints. For example, a test framework may
+generate a task with 100 test run processes, but wants to run it on
+a machine with only 4 cores. You can limit the amount of parallelism to
+4 by setting <code>max_concurrency=4</code> in your task configuration.</p>
+
+<p>For example, the following task spawns 180 Processes (&ldquo;mappers&rdquo;)
+to compute individual elements of a 180 degree sine table, all dependent
+upon one final Process (&ldquo;reducer&rdquo;) to tabulate the results:</p>
+<pre class="highlight text">def make_mapper(id):
+  return Process(
+    name = &quot;mapper%03d&quot; % id,
+    cmdline = &quot;echo &#39;scale=50;s(%d\*4\*a(1)/180)&#39; | bc -l &gt;
+               temp.sine_table.%03d&quot; % (id, id))
+
+def make_reducer():
+  return Process(name = &quot;reducer&quot;, cmdline = &quot;cat temp.\* | nl \&gt; sine\_table.txt
+                 &amp;&amp; rm -f temp.\*&quot;)
+
+processes = map(make_mapper, range(180))
+
+task = Task(
+  name = &quot;mapreduce&quot;,
+  processes = processes + [make\_reducer()],
+  constraints = [Constraint(order = [mapper.name(), &#39;reducer&#39;]) for mapper
+                 in processes],
+  max_concurrency = 8)
+</pre>
+<h4 id="finalization_wait">finalization_wait</h4>
+
+<p>Tasks have three active stages: <code>ACTIVE</code>, <code>CLEANING</code>, and <code>FINALIZING</code>. The
+<code>ACTIVE</code> stage is when ordinary processes run. This stage lasts as
+long as Processes are running and the Task is healthy. The moment either
+all Processes have finished successfully or the Task has reached a
+maximum Process failure limit, it goes into <code>CLEANING</code> stage and send
+SIGTERMs to all currently running Processes and their process trees.
+Once all Processes have terminated, the Task goes into <code>FINALIZING</code> stage
+and invokes the schedule of all Processes with the &ldquo;final&rdquo; attribute set to True.</p>
+
+<p>This whole process from the end of <code>ACTIVE</code> stage to the end of <code>FINALIZING</code>
+must happen within <code>finalization_wait</code> seconds. If it does not
+finish during that time, all remaining Processes are sent SIGKILLs
+(or if they depend upon uncompleted Processes, are
+never invoked.)</p>
+
+<p>Client applications with higher priority may force a shorter
+finalization wait (e.g. through parameters to <code>thermos kill</code>), so this
+is mostly a best-effort signal.</p>
+
+<h3 id="constraint-object">Constraint Object</h3>
+
+<p>Current constraint objects only support a single ordering constraint, <code>order</code>,
+which specifies its processes run sequentially in the order given. By
+default, all processes run in parallel when bound to a <code>Task</code> without
+ordering constraints.</p>
+
+<table><thead>
+<tr>
+<th>param</th>
+<th style="text-align: center">type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td>order</td>
+<td style="text-align: center">List of String</td>
+<td>List of processes by name (String) that should be run serially.</td>
+</tr>
+</tbody></table>
+
+<h3 id="resource-object">Resource Object</h3>
+
+<p>Specifies the amount of CPU, Ram, and disk resources the task needs. See the
+<a href="/documentation/latest/resource-isolation/">Resource Isolation document</a> for suggested values and to understand how
+resources are allocated.</p>
+
+<table><thead>
+<tr>
+<th>param</th>
+<th style="text-align: center">type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>cpu</code></td>
+<td style="text-align: center">Float</td>
+<td>Fractional number of cores required by the task.</td>
+</tr>
+<tr>
+<td><code>ram</code></td>
+<td style="text-align: center">Integer</td>
+<td>Bytes of RAM required by the task.</td>
+</tr>
+<tr>
+<td><code>disk</code></td>
+<td style="text-align: center">Integer</td>
+<td>Bytes of disk required by the task.</td>
+</tr>
+</tbody></table>
+
+<h1 id="job-schema">Job Schema</h1>
+
+<h3 id="job-objects">Job Objects</h3>
+
+<table><thead>
+<tr>
+<th>name</th>
+<th style="text-align: center">type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>task</code></td>
+<td style="text-align: center">Task</td>
+<td>The Task object to bind to this job. Required.</td>
+</tr>
+<tr>
+<td><code>name</code></td>
+<td style="text-align: center">String</td>
+<td>Job name. (Default: inherited from the task attribute&rsquo;s name)</td>
+</tr>
+<tr>
+<td><code>role</code></td>
+<td style="text-align: center">String</td>
+<td>Job role account. Required.</td>
+</tr>
+<tr>
+<td><code>cluster</code></td>
+<td style="text-align: center">String</td>
+<td>Cluster in which this job is scheduled. Required.</td>
+</tr>
+<tr>
+<td><code>environment</code></td>
+<td style="text-align: center">String</td>
+<td>Job environment, default <code>devel</code>. Must be one of <code>prod</code>, <code>devel</code>, <code>test</code> or <code>staging&lt;number&gt;</code>.</td>
+</tr>
+<tr>
+<td><code>contact</code></td>
+<td style="text-align: center">String</td>
+<td>Best email address to reach the owner of the job. For production jobs, this is usually a team mailing list.</td>
+</tr>
+<tr>
+<td><code>instances</code></td>
+<td style="text-align: center">Integer</td>
+<td>Number of instances (sometimes referred to as replicas or shards) of the task to create. (Default: 1)</td>
+</tr>
+<tr>
+<td><code>cron_schedule</code> <strong>(Present, but not supported and a no-op)</strong></td>
+<td style="text-align: center">String</td>
+<td>UTC Cron schedule in cron format. May only be used with non-service jobs. Default: None (not a cron job.)</td>
+</tr>
+<tr>
+<td><code>cron_collision_policy</code> <strong>(Present, but not supported and a no-op)</strong></td>
+<td style="text-align: center">String</td>
+<td>Policy to use when a cron job is triggered while a previous run is still active. KILL<em>EXISTING Kill the previous run, and schedule the new run CANCEL</em>NEW Let the previous run continue, and cancel the new run. RUN<em>OVERLAP Let the previous run continue, and schedule the new run. (Default: KILL</em>EXISTING)</td>
+</tr>
+<tr>
+<td><code>update_config</code></td>
+<td style="text-align: center"><code>update_config</code> object</td>
+<td>Parameters for controlling the rate and policy of rolling updates.</td>
+</tr>
+<tr>
+<td><code>constraints</code></td>
+<td style="text-align: center">dict</td>
+<td>Scheduling constraints for the tasks. See the section on the <a href="#Specifying-Scheduling-Constraints">constraint specification language</a></td>
+</tr>
+<tr>
+<td><code>service</code></td>
+<td style="text-align: center">Boolean</td>
+<td>If True, restart tasks regardless of success or failure. (Default: False)</td>
+</tr>
+<tr>
+<td><code>daemon</code></td>
+<td style="text-align: center">Boolean</td>
+<td>A DEPRECATED alias for &ldquo;service&rdquo;. (Default: False)</td>
+</tr>
+<tr>
+<td><code>max_task_failures</code></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum number of failures after which the task is considered to have failed (Default: 1) Set to -1 to allow for infinite failures</td>
+</tr>
+<tr>
+<td><code>priority</code></td>
+<td style="text-align: center">Integer</td>
+<td>Preemption priority to give the task (Default 0). Tasks with higher priorities may preempt tasks at lower priorities.</td>
+</tr>
+<tr>
+<td><code>production</code></td>
+<td style="text-align: center">Boolean</td>
+<td>Whether or not this is a production task backed by quota (Default: False). Production jobs may preempt any non-production job, and may only be preempted by production jobs in the same role and of higher priority. To run jobs at this level, the job role must have the appropriate quota.</td>
+</tr>
+<tr>
+<td><code>health_check_config</code></td>
+<td style="text-align: center"><code>heath_check_config</code> object</td>
+<td>Parameters for controlling a task&rsquo;s health checks via HTTP. Only used if a  health port was assigned with a command line wildcard.</td>
+</tr>
+</tbody></table>
+
+<h3 id="services">Services</h3>
+
+<p>Jobs with the <code>service</code> flag set to True are called Services. The <code>Service</code>
+alias can be used as shorthand for <code>Job</code> with <code>service=True</code>.
+Services are differentiated from non-service Jobs in that tasks
+always restart on completion, whether successful or unsuccessful.
+Jobs without the service bit set only restart up to
+<code>max_task_failures</code> times and only if they terminated unsuccessfully
+either due to human error or machine failure.</p>
+
+<h3 id="updateconfig-objects">UpdateConfig Objects</h3>
+
+<p>Parameters for controlling the rate and policy of rolling updates.</p>
+
+<table><thead>
+<tr>
+<th>object</th>
+<th style="text-align: center">type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>batch_size</code></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum number of shards to be updated in one iteration (Default: 1)</td>
+</tr>
+<tr>
+<td><code>restart_threshold</code></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum number of seconds before a shard must move into the <code>RUNNING</code> state before considered a failure (Default: 60)</td>
+</tr>
+<tr>
+<td><code>watch_secs</code></td>
+<td style="text-align: center">Integer</td>
+<td>Minimum number of seconds a shard must remain in <code>RUNNING</code> state before considered a success (Default: 30)</td>
+</tr>
+<tr>
+<td><code>max_per_shard_failures</code></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum number of restarts per shard during update. Increments total failure count when this limit is exceeded. (Default: 0)</td>
+</tr>
+<tr>
+<td><code>max_total_failures</code></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum number of shard failures to be tolerated in total during an update. Cannot be greater than or equal to the total number of tasks in a job. (Default: 0)</td>
+</tr>
+</tbody></table>
+
+<h3 id="healthcheckconfig-objects">HealthCheckConfig Objects</h3>
+
+<p>Parameters for controlling a task&rsquo;s health checks via HTTP.</p>
+
+<table><thead>
+<tr>
+<th>object</th>
+<th style="text-align: center">type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>initial_interval_secs</code></td>
+<td style="text-align: center">Integer</td>
+<td>Initial delay for performing an HTTP health check. (Default: 60)</td>
+</tr>
+<tr>
+<td><code>interval_secs</code></td>
+<td style="text-align: center">Integer</td>
+<td>Interval on which to check the task&rsquo;s health via HTTP. (Default: 30)</td>
+</tr>
+<tr>
+<td><code>timeout_secs</code></td>
+<td style="text-align: center">Integer</td>
+<td>HTTP request timeout. (Default: 1)</td>
+</tr>
+<tr>
+<td><code>max_consecutive_failures</code></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum number of consecutive failures that tolerated before considering a task unhealthy (Default: 0)</td>
+</tr>
+</tbody></table>
+
+<h1 id="specifying-scheduling-constraints">Specifying Scheduling Constraints</h1>
+
+<p>Most users will not need to specify constraints explicitly, as the
+scheduler automatically inserts reasonable defaults that attempt to
+ensure reliability without impacting schedulability. For example, the
+scheduler inserts a <code>host: limit:1</code> constraint, ensuring
+that your shards run on different physical machines. Please do not
+set this field unless you are sure of what you are doing.</p>
+
+<p>In the <code>Job</code> object there is a map <code>constraints</code> from String to String
+allowing the user to tailor the schedulability of tasks within the job.</p>
+
+<p>Each slave in the cluster is assigned a set of string-valued
+key/value pairs called attributes. For example, consider the host
+<code>cluster1-aaa-03-sr2</code> and its following attributes (given in key:value
+format): <code>host:cluster1-aaa-03-sr2</code> and <code>rack:aaa</code>.</p>
+
+<p>The constraint map&rsquo;s key value is the attribute name in which we
+constrain Tasks within our Job. The value is how we constrain them.
+There are two types of constraints: <em>limit constraints</em> and <em>value
+constraints</em>.</p>
+
+<table><thead>
+<tr>
+<th>constraint</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td>Limit</td>
+<td>A string that specifies a limit for a constraint. Starts with <code>&#39;limit:</code> followed by an Integer and closing single quote, such as <code>&#39;limit:1&#39;</code>.</td>
+</tr>
+<tr>
+<td>Value</td>
+<td>A string that specifies a value for a constraint. To include a list of values, separate the values using commas. To negate the values of a constraint, start with a <code>!</code> <code>.</code></td>
+</tr>
+</tbody></table>
+
+<p>You can also control machine diversity using constraints. The below
+constraint ensures that no more than two instances of your job may run
+on a single host. Think of this as a &ldquo;group by&rdquo; limit.</p>
+<pre class="highlight text">constraints = {
+  &#39;host&#39;: &#39;limit:2&#39;,
+}
+</pre>
+<p>Likewise, you can use constraints to control rack diversity, e.g. at
+most one task per rack:</p>
+<pre class="highlight text">constraints = {
+  &#39;rack&#39;: &#39;limit:1&#39;,
+}
+</pre>
+<p>Use these constraints sparingly as they can dramatically reduce Tasks&#39; schedulability.</p>
+
+<h1 id="template-namespaces">Template Namespaces</h1>
+
+<p>Currently, a few Pystachio namespaces have special semantics. Using them
+in your configuration allow you to tailor application behavior
+through environment introspection or interact in special ways with the
+Aurora client or Aurora-provided services.</p>
+
+<h3 id="mesos-namespace">mesos Namespace</h3>
+
+<p>The <code>mesos</code> namespace contains the <code>instance</code> variable that can be used
+to distinguish between Task replicas.</p>
+
+<table><thead>
+<tr>
+<th>variable name</th>
+<th style="text-align: center">type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>instance</code></td>
+<td style="text-align: center">Integer</td>
+<td>The instance number of the created task. A job with 5 replicas has instance numbers 0, 1, 2, 3, and 4.</td>
+</tr>
+</tbody></table>
+
+<h3 id="thermos-namespace">thermos Namespace</h3>
+
+<p>The <code>thermos</code> namespace contains variables that work directly on the
+Thermos platform in addition to Aurora. This namespace is fully
+compatible with Tasks invoked via the <code>thermos</code> CLI.</p>
+
+<table><thead>
+<tr>
+<th style="text-align: center">variable</th>
+<th>type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td style="text-align: center"><code>ports</code></td>
+<td>map of string to Integer</td>
+<td>A map of names to port numbers</td>
+</tr>
+<tr>
+<td style="text-align: center"><code>task_id</code></td>
+<td>string</td>
+<td>The task ID assigned to this task.</td>
+</tr>
+</tbody></table>
+
+<p>The <code>thermos.ports</code> namespace is automatically populated by Aurora when
+invoking tasks on Mesos. When running the <code>thermos</code> command directly,
+these ports must be explicitly mapped with the <code>-P</code> option.</p>
+
+<p>For example, if &lsquo;{{<code>thermos.ports[http]</code>}}&rsquo; is specified in a <code>Process</code>
+configuration, it is automatically extracted and auto-populated by
+Aurora, but must be specified with, for example, <code>thermos -P http:12345</code>
+to map <code>http</code> to port 12345 when running via the CLI.</p>
+
+<h1 id="basic-examples">Basic Examples</h1>
+
+<p>These are provided to give a basic understanding of simple Aurora jobs.</p>
+
+<h3 id="hello_world.aurora">hello_world.aurora</h3>
+
+<p>Put the following in a file named <code>hello_world.aurora</code>, substituting your own values
+for values such as <code>cluster</code>s.</p>
+<pre class="highlight text">import os
+hello_world_process = Process(name = &#39;hello_world&#39;, cmdline = &#39;echo hello world&#39;)
+
+hello_world_task = Task(
+  resources = Resources(cpu = 0.1, ram = 16 * MB, disk = 16 * MB),
+  processes = [hello_world_process])
+
+hello_world_job = Job(
+  cluster = &#39;cluster1&#39;,
+  role = os.getenv(&#39;USER&#39;),
+  task = hello_world_task)
+
+jobs = [hello_world_job]
+</pre>
+<p>Then issue the following commands to create and kill the job, using your own values for the job key.</p>
+<pre class="highlight text">aurora create cluster1/$USER/test/hello_world hello_world.aurora
+
+aurora kill cluster1/$USER/test/hello_world
+</pre>
+<h3 id="environment-tailoring">Environment Tailoring</h3>
+
+<h4 id="helloworldproductionized.aurora">hello<em>world</em>productionized.aurora</h4>
+
+<p>Put the following in a file named <code>hello_world_productionized.aurora</code>, substituting your own values
+for values such as <code>cluster</code>s.</p>
+<pre class="highlight text">include(&#39;hello_world.aurora&#39;)
+
+production_resources = Resources(cpu = 1.0, ram = 512 * MB, disk = 2 * GB)
+staging_resources = Resources(cpu = 0.1, ram = 32 * MB, disk = 512 * MB)
+hello_world_template = hello_world(
+    name = &quot;hello_world-{{cluster}}&quot;
+    task = hello_world(resources=production_resources))
+
+jobs = [
+  # production jobs
+  hello_world_template(cluster = &#39;cluster1&#39;, instances = 25),
+  hello_world_template(cluster = &#39;cluster2&#39;, instances = 15),
+
+  # staging jobs
+  hello_world_template(
+    cluster = &#39;local&#39;,
+    instances = 1,
+    task = hello_world(resources=staging_resources)),
+]
+</pre>
+<p>Then issue the following commands to create and kill the job, using your own values for the job key</p>
+<pre class="highlight text">aurora create cluster1/$USER/test/hello_world-cluster1 hello_world_productionized.aurora
+
+aurora kill cluster1/$USER/test/hello_world-cluster1
+</pre>
+	  </div>
+      <div class="container">
+    <hr>
+    <footer class="footer">
+        <div class="row-fluid">
+            <div class="span2 text-left">
+                <h3>Links</h3>
+                <ul class="unstyled">
+                    <li><a href="/downloads/">Downloads</a></li>
+                    <li><a href="/developers/">Developers</a></li>                    
+                </ul>
+            </div>
+            <div class="span3 text-left">
+                <h3>Community</h3>
+                <ul class="unstyled">
+                    <li><a href="/community/">Mailing Lists</a></li>
+                    <li><a href="http://issues.apache.org/jira/browse/aurora">Issue Tracking</a></li>
+                    <li><a href="/docs/howtocontribute/">How To Contribute</a></li>
+                </ul>
+            </div>
+            <div class="span7 text-left">
+            	<h3>Apache Software Foundation</h3>
+
+							<div class="span8">
+                Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. Apache, Apache Thrift, and the Apache feather logo are trademarks of The Apache Software Foundation. Currently part of the <a href="http://incubator.apache.org">Apache Incubator</a>.
+							</div>
+							<div class=" pull-right">
+								<a href="http://incubator.apache.org" class="logo"><img src="/assets/img/apache_incubator_logo.png" alt="Apache Incubator" class="pull-right"/></a>
+							</div>
+            </div>
+
+        </div>
+
+    </footer>
+</div>
+
+	</body>
+</html>
+



Mime
View raw message