aurora-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dles...@apache.org
Subject svn commit: r1659198 - in /incubator/aurora/site: ./ publish/documentation/latest/ publish/documentation/latest/configuration-reference/ publish/documentation/latest/configuration-tutorial/ publish/documentation/latest/contributing/ publish/documentati...
Date Thu, 12 Feb 2015 10:18:53 GMT
Author: dlester
Date: Thu Feb 12 10:18:52 2015
New Revision: 1659198

URL: http://svn.apache.org/r1659198
Log:
Updates rakefile to properly move contributing.md for web rendering, and updates website documentation.

Modified:
    incubator/aurora/site/Rakefile
    incubator/aurora/site/publish/documentation/latest/configuration-reference/index.html
    incubator/aurora/site/publish/documentation/latest/configuration-tutorial/index.html
    incubator/aurora/site/publish/documentation/latest/contributing/index.html
    incubator/aurora/site/publish/documentation/latest/deploying-aurora-scheduler/index.html
    incubator/aurora/site/publish/documentation/latest/developing-aurora-client/index.html
    incubator/aurora/site/publish/documentation/latest/hooks/index.html
    incubator/aurora/site/publish/documentation/latest/index.html
    incubator/aurora/site/publish/documentation/latest/vagrant/index.html
    incubator/aurora/site/source/documentation/latest.html.md
    incubator/aurora/site/source/documentation/latest/configuration-reference.md
    incubator/aurora/site/source/documentation/latest/configuration-tutorial.md
    incubator/aurora/site/source/documentation/latest/contributing.md
    incubator/aurora/site/source/documentation/latest/deploying-aurora-scheduler.md
    incubator/aurora/site/source/documentation/latest/developing-aurora-client.md
    incubator/aurora/site/source/documentation/latest/hooks.md
    incubator/aurora/site/source/documentation/latest/vagrant.md

Modified: incubator/aurora/site/Rakefile
URL: http://svn.apache.org/viewvc/incubator/aurora/site/Rakefile?rev=1659198&r1=1659197&r2=1659198&view=diff
==============================================================================
--- incubator/aurora/site/Rakefile (original)
+++ incubator/aurora/site/Rakefile Thu Feb 12 10:18:52 2015
@@ -32,6 +32,7 @@ task :update_docs do
   FileUtils.rm_f(Dir.glob("source/documentation/latest/*.md"))
   FileUtils.cp_r(Dir.glob("#{tmp_dir}/aurora/docs/images/"), File.expand_path("source/documentation/latest/"))
   FileUtils.cp_r(Dir.glob("#{tmp_dir}/aurora/docs/*.md"), File.expand_path("source/documentation/latest/"))
+  FileUtils.cp_r("#{tmp_dir}/aurora/CONTRIBUTING.md", "source/documentation/latest/contributing.md")
   puts "Parse documentation files to removing md extension in any links"
   Dir.chdir("#{docs_dir}/latest/") {
     Dir.glob('*.md').each { |doc|

Modified: incubator/aurora/site/publish/documentation/latest/configuration-reference/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/configuration-reference/index.html?rev=1659198&r1=1659197&r2=1659198&view=diff
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/configuration-reference/index.html (original)
+++ incubator/aurora/site/publish/documentation/latest/configuration-reference/index.html Thu Feb 12 10:18:52 2015
@@ -84,6 +84,7 @@
 <li><a href="#updateconfig-objects">UpdateConfig Objects</a></li>
 <li><a href="#healthcheckconfig-objects">HealthCheckConfig Objects</a></li>
 <li><a href="#announcer-objects">Announcer Objects</a></li>
+<li><a href="#container">Container Objects</a></li>
 </ul></li>
 <li><a href="#specifying-scheduling-constraints">Specifying Scheduling Constraints</a></li>
 <li><a href="#template-namespaces">Template Namespaces</a>
@@ -545,6 +546,11 @@ resources are allocated.</p>
 <td style="text-align: center"><code>heath_check_config</code> object</td>
 <td>Parameters for controlling a task&rsquo;s health checks via HTTP. Only used if a  health port was assigned with a command line wildcard.</td>
 </tr>
+<tr>
+<td><code>container</code></td>
+<td style="text-align: center"><code>Container</code> object</td>
+<td>An optional container to run all processes inside of.</td>
+</tr>
 </tbody></table>
 
 <h3 id="services">Services</h3>
@@ -678,6 +684,43 @@ tasks with the same static port allocati
 External constraints such as slave attributes should be used to enforce such
 guarantees should they be needed.</p>
 
+<h3 id="container-object">Container Object</h3>
+
+<p><em>Note: The only container type currently supported is &ldquo;docker&rdquo;.  Docker support is currently EXPERIMENTAL.</em>
+<em>Note: In order to correctly execute processes inside a job, the Docker container must have python 2.7 installed.</em></p>
+
+<p>Describes the container the job&rsquo;s processes will run inside.</p>
+
+<table><thead>
+<tr>
+<th>param</th>
+<th style="text-align: center">type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>docker</code></td>
+<td style="text-align: center">Docker</td>
+<td>A docker container to use.</td>
+</tr>
+</tbody></table>
+
+<h3 id="docker-object">Docker Object</h3>
+
+<table><thead>
+<tr>
+<th>param</th>
+<th style="text-align: center">type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>image</code></td>
+<td style="text-align: center">String</td>
+<td>The name of the docker image to execute.  If the image does not exist locally it will be pulled with <code>docker pull</code>.</td>
+</tr>
+</tbody></table>
+
 <h1 id="specifying-scheduling-constraints">Specifying Scheduling Constraints</h1>
 
 <p>Most users will not need to specify constraints explicitly, as the

Modified: incubator/aurora/site/publish/documentation/latest/configuration-tutorial/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/configuration-tutorial/index.html?rev=1659198&r1=1659197&r2=1659198&view=diff
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/configuration-tutorial/index.html (original)
+++ incubator/aurora/site/publish/documentation/latest/configuration-tutorial/index.html Thu Feb 12 10:18:52 2015
@@ -754,25 +754,6 @@ replaced with nothing. If we add a bindi
 <p>We&rsquo;d get back:</p>
 <pre class="highlight text">Hello Tom
 </pre>
-<p>We can also use {{}} variables as sectional variables. Let&rsquo;s say we
-have:</p>
-<pre class="highlight text">{{#x}} Testing... {{/x}}
-</pre>
-<p>If <code>x</code> evaluates to <code>True</code>, the text between the sectional tags is
-shown. If there is no value for <code>x</code> or it evaluates to <code>False</code>, the
-between tags text is not shown. So, at a basic level, a sectional
-variable acts as a conditional.</p>
-
-<p>However, if the sectional variable evaluates to a list, array, etc. it
-acts as a <code>foreach</code>. For example,</p>
-<pre class="highlight text">{{#x}} {{name}} {{/x}}
-</pre>
-<p>with</p>
-<pre class="highlight json"><span class="p">{</span><span class="w"> </span><span class="s2">&quot;x&quot;</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="s2">&quot;name&quot;</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="s2">&quot;tic&quot;</span><span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="s2">&quot;name&quot;</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="s2">&quot;tac&quot;</span><span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="s2">&quot;name&quot;</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="s2">&quot;toe&quot;</span><span class="w"> </span><span class="p">}</span
 ><span class="w"> </span><span class="p">]</span><span class="w"> </span><span class="p">}</span><span class="w">
-</span></pre>
-<p>evaluates to</p>
-<pre class="highlight text">tic tac toe
-</pre>
 <p>Every Pystachio object has an associated <code>.bind</code> method that can bind
 values to {{}} variables. Bindings are not immediately evaluated.
 Instead, they are evaluated only when the interpolated value of the

Modified: incubator/aurora/site/publish/documentation/latest/contributing/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/contributing/index.html?rev=1659198&r1=1659197&r2=1659198&view=diff
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/contributing/index.html (original)
+++ incubator/aurora/site/publish/documentation/latest/contributing/index.html Thu Feb 12 10:18:52 2015
@@ -57,7 +57,7 @@ both languages.</p>
 <h2 id="find-something-to-do">Find Something to Do</h2>
 
 <p>There are issues in <a href="https://issues.apache.org/jira/browse/AURORA">Jira</a> with the
-<a href="https://issues.apache.org/jira/browse/AURORA-189?jql=project%20%3D%20AURORA%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20newbie%20ORDER%20BY%20priority%20DESC">&ldquo;newbie&rdquo; tag</a>
+<a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20AURORA%20AND%20labels%20%3D%20newbie%20and%20resolution%3Dunresolved">&ldquo;newbie&rdquo; label</a>
 that are good starting places for new Aurora contributors; pick one of these and dive in! Once
 you&rsquo;ve got a patch, the next step is to post a review.</p>
 

Modified: incubator/aurora/site/publish/documentation/latest/deploying-aurora-scheduler/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/deploying-aurora-scheduler/index.html?rev=1659198&r1=1659197&r2=1659198&view=diff
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/deploying-aurora-scheduler/index.html (original)
+++ incubator/aurora/site/publish/documentation/latest/deploying-aurora-scheduler/index.html Thu Feb 12 10:18:52 2015
@@ -59,6 +59,7 @@ machines.  This guide helps you get the
 <li><a href="#initializing-the-replicated-log">Initializing the Replicated Log</a></li>
 <li><a href="#storage-performance-considerations">Storage Performance Considerations</a></li>
 <li><a href="#network-considerations">Network considerations</a></li>
+<li><a href="#considerations-for-running-jobs-in-docker">Considerations for running jobs in docker</a></li>
 </ul></li>
 <li><a href="#running-aurora">Running Aurora</a>
 
@@ -92,7 +93,7 @@ machines.  This guide helps you get the
 
 <p>The Aurora scheduler is a standalone Java server. As part of the build process it creates a bundle
 of all its dependencies, with the notable exceptions of the JVM and libmesos. Each target server
-should have a JVM (Java 7 or higher) and libmesos (0.20.1) installed.</p>
+should have a JVM (Java 7 or higher) and libmesos (0.21.1) installed.</p>
 
 <h3 id="creating-the-distribution-.zip-file-(optional)">Creating the Distribution .zip File (Optional)</h3>
 
@@ -219,6 +220,24 @@ AURORA_FLAGS=(
 export LIBPROCESS_PORT=8083
 # ...
 </pre>
+<h3 id="considerations-for-running-jobs-in-docker-containers">Considerations for running jobs in docker containers</h3>
+
+<p><em>Note: Docker support is currently EXPERIMENTAL.</em></p>
+
+<p>In order for Aurora to launch jobs using docker containers, a few extra configuration options
+must be set.  The <a href="http://mesos.apache.org/documentation/latest/docker-containerizer/">docker containerizer</a>
+must be enabled on the mesos slaves by launching them with the <code>--containerizers=docker,mesos</code> option.</p>
+
+<p>By default, aurora will configure mesos to copy the file specified in <code>-thermos_executor_path</code>
+into the container&rsquo;s sandbox.  If using a wrapper script to launch the thermos executor,
+specify the path to the wrapper in that arguement.  In addition, the executor pex itself must be
+included in <code>-thermos_executor_resources</code> option.  Doing so will ensure that both the wrapper script
+and executor are correctly copied into the sandbox.  In addition, ensure the wrapper script does not
+access resources outside of the sandbox, because when running inside a docker container they will not
+exist.</p>
+
+<p>In order to correctly execute processes inside a job, the docker container must have python 2.7 installed.</p>
+
 <h2 id="running-aurora">Running Aurora</h2>
 
 <p>Configure a supervisor like <a href="http://mmonit.com/monit/">Monit</a> or
@@ -279,7 +298,7 @@ enforce this.</p>
 )
 </pre>
 <p>The job configuration is indicating that it should only be scheduled on slaves with the attribute
-<code>dedicated:dba_team/redis</code>.  Additionally, Aurora will prevent any tasks that do <em>not</em> have that
+<code>dedicated:db_team/redis</code>.  Additionally, Aurora will prevent any tasks that do <em>not</em> have that
 constraint from running on those slaves.</p>
 
 <h2 id="common-problems">Common problems</h2>

Modified: incubator/aurora/site/publish/documentation/latest/developing-aurora-client/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/developing-aurora-client/index.html?rev=1659198&r1=1659197&r2=1659198&view=diff
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/developing-aurora-client/index.html (original)
+++ incubator/aurora/site/publish/documentation/latest/developing-aurora-client/index.html Thu Feb 12 10:18:52 2015
@@ -98,11 +98,11 @@ Once the project is loaded:
 <p>After following these instructions, you should now be able to run/debug tests directly from the IDE
 by right-clicking on a test (or test class) and choosing to run or debug:</p>
 
-<p><a href="images/debug-client-test.png"><img src="images/debug-client-test.png" width="697" height="444" /></a></p>
+<p><a href="/documentation/latest/images/debug-client-test.png"><img alt="Debug Client Test" src="../images/debug-client-test.png" /></a></p>
 
 <p>If you&rsquo;ve set a breakpoint, you can see the run will now stop and let you debug:</p>
 
-<p><a href="images/debugging-client-test.png"><img src="images/debugging-client-test.png" width="697" height="444" /></a></p>
+<p><a href="/documentation/latest/images/debugging-client-test.png"><img alt="Debugging Client Test" src="../images/debugging-client-test.png" /></a></p>
 
 <h3 id="running/debugging-the-client">Running/Debugging the Client</h3>
 

Modified: incubator/aurora/site/publish/documentation/latest/hooks/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/hooks/index.html?rev=1659198&r1=1659197&r2=1659198&view=diff
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/hooks/index.html (original)
+++ incubator/aurora/site/publish/documentation/latest/hooks/index.html Thu Feb 12 10:18:52 2015
@@ -141,27 +141,37 @@ returning <code>True</code>. For example
 <tr>
 <td><code>cancel_update</code></td>
 <td><code>self</code>, <code>job_key</code></td>
-<td><code>cancel_update</code></td>
+<td><code>job cancel-update</code></td>
 </tr>
 <tr>
 <td><code>create_job</code></td>
 <td><code>self</code>, <code>config</code></td>
-<td><code>create</code>, <code>runtask</td>
+<td><code>job create</code>, <code>runtask</td>
 </tr>
 <tr>
 <td><code>restart</code></td>
 <td><code>self</code>, <code>job_key</code>, <code>shards</code>, <code>update_config</code>, <code>health_check_interval_seconds</code></td>
-<td><code>restart</code></td>
+<td><code>job restart</code></td>
 </tr>
 <tr>
 <td><code>update_job</code></td>
 <td><code>self</code>, <code>config</code>, <code>health_check_interval_seconds=3</code>, <code>shards=None</code></td>
-<td><code>update</code></td>
+<td><code>job update</code></td>
 </tr>
 <tr>
 <td><code>kill_job</code></td>
 <td><code>self</code>, <code>job_key</code>, <code>shards=None</code></td>
-<td><code>kill</code></td>
+<td><code>job kill</code></td>
+</tr>
+<tr>
+<td><code>start_cronjob</code></td>
+<td><code>self</code>, <code>job_key</code></td>
+<td><code>cron start</code></td>
+</tr>
+<tr>
+<td><code>start_job_update</code></td>
+<td><code>self</code>, <code>config</code>, <code>instances=None</code></td>
+<td><code>beta-update start</code></td>
 </tr>
 </tbody></table>
 
@@ -250,7 +260,7 @@ returning <code>True</code>. For example
 
 <h2 id="generic-hooks">Generic Hooks</h2>
 
-<p>There are five Aurora API Methods which any of the three hook types can attach to. Thus, there are 15 possible hook/method combinations for a single <code>.aurora</code> config file. Say that you define <code>pre_</code> and <code>post_</code> hooks for the <code>restart</code> method. That leaves 13 undefined hook/method combinations; <code>err_restart</code> and the 3 <code>pre_</code>, <code>post_</code>, and <code>err_</code> hooks for each of the other 4 hookable methods. You can define what happens when any of these otherwise undefined 13 hooks execute via a generic hook, whose signature is:</p>
+<p>There are seven Aurora API Methods which any of the three hook types can attach to. Thus, there are 21 possible hook/method combinations for a single <code>.aurora</code> config file. Say that you define <code>pre_</code> and <code>post_</code> hooks for the <code>restart</code> method. That leaves 19 undefined hook/method combinations; <code>err_restart</code> and the 3 <code>pre_</code>, <code>post_</code>, and <code>err_</code> hooks for each of the other 6 hookable methods. You can define what happens when any of these otherwise undefined 19 hooks execute via a generic hook, whose signature is:</p>
 <pre class="highlight python"><span class="n">generic_hook</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">hook_config</span><span class="p">,</span> <span class="n">event</span><span class="p">,</span> <span class="n">method_name</span><span class="p">,</span> <span class="n">result_or_err</span><span class="p">,</span> <span class="n">args</span><span class="o">*</span><span class="p">,</span> <span class="n">kw</span><span class="o">**</span><span class="p">)</span>
 </pre>
 <p>where:</p>

Modified: incubator/aurora/site/publish/documentation/latest/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/index.html?rev=1659198&r1=1659197&r2=1659198&view=diff
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/index.html (original)
+++ incubator/aurora/site/publish/documentation/latest/index.html Thu Feb 12 10:18:52 2015
@@ -49,7 +49,7 @@
 <li>Developers: All the information you need to start modifying Aurora and contributing back to the project.</li>
 </ul>
 
-<p>This documentation is a work in progress, and we encourage you to ask questions on the <a href="http://aurora.incubator.apache.org/community/">Aurora developer list</a> or the <code>#aurora</code> IRC channel on <code>irc.freenode.net</code>.</p>
+<p>We encourage you to ask questions on the <a href="http://aurora.incubator.apache.org/community/">Aurora developer list</a> or the <code>#aurora</code> IRC channel on <code>irc.freenode.net</code>.</p>
 
 <h2 id="users">Users</h2>
 

Modified: incubator/aurora/site/publish/documentation/latest/vagrant/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/vagrant/index.html?rev=1659198&r1=1659197&r2=1659198&view=diff
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/vagrant/index.html (original)
+++ incubator/aurora/site/publish/documentation/latest/vagrant/index.html Thu Feb 12 10:18:52 2015
@@ -41,49 +41,116 @@
 <h5 class="page-header text-uppercase">Documentation</h5>
 <h1 id="getting-started">Getting Started</h1>
 
-<p>To replicate a real cluster environment as closely as possible, we use
-<a href="http://www.vagrantup.com/">Vagrant</a> to launch a complete Aurora cluster in a virtual machine.</p>
+<p>This document shows you how to configure a complete cluster using a virtual machine. This setup
+replicates a real cluster in your development machine as closely as possible. After you complete
+the steps outlined here, you will be ready to create and run your first Aurora job.</p>
+
+<p>The following sections describe these steps in detail:</p>
+
+<ol>
+<li><a href="#user-content-overview">Overview</a></li>
+<li><a href="#user-content-install-virtualbox-and-vagrant">Install VirtualBox and Vagrant</a></li>
+<li><a href="#user-content-clone-the-aurora-repository">Clone the Aurora repository</a></li>
+<li><a href="#user-content-start-the-local-cluster">Start the local cluster</a></li>
+<li><a href="#user-content-log-onto-the-vm">Log onto the VM</a></li>
+<li><a href="#user-content-run-your-first-job">Run your first job</a></li>
+<li><a href="#user-content-rebuild-components">Rebuild components</a></li>
+<li><a href="#user-content-shut-down-or-delete-your-local-cluster">Shut down or delete your local cluster</a></li>
+<li><a href="#user-content-troubleshooting">Troubleshooting</a></li>
+</ol>
+
+<h2 id="overview">Overview</h2>
+
+<p>The Aurora distribution includes a set of scripts that enable you to create a local cluster in
+your development machine. These scripts use <a href="https://www.vagrantup.com/">Vagrant</a> and
+<a href="https://www.virtualbox.org/">VirtualBox</a> to run and configure a virtual machine. Once the
+virtual machine is running, the scripts install and initialize Aurora and any required components
+to create the local cluster.</p>
+
+<h2 id="install-virtualbox-and-vagrant">Install VirtualBox and Vagrant</h2>
+
+<p>First, download and install <a href="https://www.virtualbox.org/">VirtualBox</a> on your development machine.</p>
+
+<p>Then download and install <a href="https://www.vagrantup.com/">Vagrant</a>. To verify that the installation
+was successful, open a terminal window and type the <code>vagrant</code> command. You should see a list of
+common commands for this tool.</p>
 
-<h2 id="prerequisites">Prerequisites</h2>
+<h2 id="clone-the-aurora-repository">Clone the Aurora repository</h2>
+
+<p>To obtain the Aurora source distribution, clone its Git repository using the following command:</p>
+<pre class="highlight text"> git clone http://git.apache.org/incubator-aurora.git
+</pre>
+<h2 id="start-the-local-cluster">Start the local cluster</h2>
+
+<p>Now change into the <code>incubator-aurora/</code> directory, which contains the Aurora source code and
+other scripts and tools:</p>
+<pre class="highlight text"> cd incubator-aurora/
+</pre>
+<p>To start the local cluster, type the following command:</p>
+<pre class="highlight text"> vagrant up
+</pre>
+<p>This command uses the configuration scripts in the Aurora distribution to:</p>
 
 <ul>
-<li><a href="https://www.virtualbox.org/">VirtualBox</a></li>
-<li><a href="http://www.vagrantup.com/">Vagrant</a></li>
-<li>A clone of the Aurora repository, or source distribution.</li>
+<li>Download a Linux system image.</li>
+<li>Start a virtual machine (VM) and configure it.</li>
+<li>Install the required build tools on the VM.</li>
+<li>Install Aurora&rsquo;s requirements (like <a href="http://mesos.apache.org/">Mesos</a> and
+<a href="http://zookeeper.apache.org/">Zookeeper</a>) on the VM.</li>
+<li>Build and install Aurora from source on the VM.</li>
+<li>Start Aurora&rsquo;s services on the VM.</li>
 </ul>
 
-<p>You can start a local cluster by running:</p>
-<pre class="highlight text">vagrant up
-</pre>
-<p>Once started, several services should be running:</p>
+<p>This process takes several minutes to complete.</p>
+
+<p>To verify that Aurora is running on the cluster, visit the following URLs:</p>
 
 <ul>
-<li>scheduler is listening on <a href="http://192.168.33.7:8081">http://192.168.33.7:8081</a></li>
-<li>observer is listening on <a href="http://192.168.33.7:1338">http://192.168.33.7:1338</a></li>
-<li>master is listening on <a href="http://192.168.33.7:5050">http://192.168.33.7:5050</a></li>
-<li>slave is listening on <a href="http://192.168.33.7:5051">http://192.168.33.7:5051</a></li>
+<li>Scheduler - <a href="http://192.168.33.7:8081">http://192.168.33.7:8081</a></li>
+<li>Observer - <a href="http://192.168.33.7:1338">http://192.168.33.7:1338</a></li>
+<li>Mesos Master - <a href="http://192.168.33.7:5050">http://192.168.33.7:5050</a></li>
+<li>Mesos Slave - <a href="http://192.168.33.7:5051">http://192.168.33.7:5051</a></li>
 </ul>
 
-<p>You can SSH into the machine with <code>vagrant ssh</code> and execute aurora client commands using the
-<code>aurora</code> command.  A pre-installed <code>clusters.json</code> file refers to your local cluster as
-<code>devcluster</code>, which you will use in client commands.</p>
+<h2 id="log-onto-the-vm">Log onto the VM</h2>
 
-<h1 id="deleting-your-local-cluster">Deleting your local cluster</h1>
+<p>To SSH into the VM, run the following command in your development machine:</p>
+<pre class="highlight text"> vagrant ssh
+</pre>
+<p>To verify that Aurora is installed in the VM, type the <code>aurora</code> command. You should see a list
+of arguments and possible commands.</p>
 
-<p>Once you are finished with your local cluster, or if you would otherwise like to start from scratch,
-you can use the command <code>vagrant destroy</code> to turn off and delete the virtual file system.</p>
+<p>The <code>/vagrant</code> directory on the VM is mapped to the <code>incubator-aurora/</code> local directory
+from which you started the cluster. You can edit files inside this directory in your development
+machine and access them from the VM under <code>/vagrant</code>.</p>
+
+<p>A pre-installed <code>clusters.json</code> file refers to your local cluster as <code>devcluster</code>, which you
+will use in client commands.</p>
+
+<h2 id="run-your-first-job">Run your first job</h2>
 
-<h1 id="rebuilding-components">Rebuilding components</h1>
+<p>Now that your cluster is up and running, you are ready to define and run your first job in Aurora.
+For more information, see the <a href="/documentation/latest/tutorial/">Aurora Tutorial</a>.</p>
+
+<h2 id="rebuild-components">Rebuild components</h2>
 
 <p>If you are changing Aurora code and would like to rebuild a component, you can use the <code>aurorabuild</code>
-command on your vagrant machine to build and restart a component.  This is considerably faster than
-destroying and rebuilding your VM.</p>
+command on the VM to build and restart a component.  This is considerably faster than destroying
+and rebuilding your VM.</p>
 
-<p><code>aurorabuild</code> accepts a list of components to build and update.  You may invoke the command with
-no arguments to get a list of supported components.</p>
+<p><code>aurorabuild</code> accepts a list of components to build and update. To get a list of supported
+components, invoke the <code>aurorabuild</code> command with no arguments:</p>
 <pre class="highlight text"> vagrant ssh -c &#39;aurorabuild client&#39;
 </pre>
-<h1 id="troubleshooting">Troubleshooting</h1>
+<h2 id="shut-down-or-delete-your-local-cluster">Shut down or delete your local cluster</h2>
+
+<p>To shut down your local cluster, run the <code>vagrant halt</code> command in your development machine. To
+start it again, run the <code>vagrant up</code> command.</p>
+
+<p>Once you are finished with your local cluster, or if you would otherwise like to start from scratch,
+you can use the command <code>vagrant destroy</code> to turn off and delete the virtual file system.</p>
+
+<h2 id="troubleshooting">Troubleshooting</h2>
 
 <p>Most of the vagrant related problems can be fixed by the following steps:</p>
 

Modified: incubator/aurora/site/source/documentation/latest.html.md
URL: http://svn.apache.org/viewvc/incubator/aurora/site/source/documentation/latest.html.md?rev=1659198&r1=1659197&r2=1659198&view=diff
==============================================================================
--- incubator/aurora/site/source/documentation/latest.html.md (original)
+++ incubator/aurora/site/source/documentation/latest.html.md Thu Feb 12 10:18:52 2015
@@ -5,7 +5,7 @@ Apache Aurora is a service scheduler tha
  * Operators: For those that wish to manage and fine-tune an Aurora cluster.
  * Developers: All the information you need to start modifying Aurora and contributing back to the project.
 
-This documentation is a work in progress, and we encourage you to ask questions on the [Aurora developer list](http://aurora.incubator.apache.org/community/) or the `#aurora` IRC channel on `irc.freenode.net`.
+We encourage you to ask questions on the [Aurora developer list](http://aurora.incubator.apache.org/community/) or the `#aurora` IRC channel on `irc.freenode.net`.
 
 ## Users
  * [Install Aurora on virtual machines on your private machine](/documentation/latest/vagrant/)
@@ -30,4 +30,4 @@ This documentation is a work in progress
  * [Contributing to the project](/documentation/latest/contributing/)
  * [Developing the Aurora Scheduler](/documentation/latest/developing-aurora-scheduler/)
  * [Developing the Aurora Client](/documentation/latest/developing-aurora-client/)
- * [Committers Guide](/documentation/latest/committers/)
\ No newline at end of file
+ * [Committers Guide](/documentation/latest/committers/)

Modified: incubator/aurora/site/source/documentation/latest/configuration-reference.md
URL: http://svn.apache.org/viewvc/incubator/aurora/site/source/documentation/latest/configuration-reference.md?rev=1659198&r1=1659197&r2=1659198&view=diff
==============================================================================
--- incubator/aurora/site/source/documentation/latest/configuration-reference.md (original)
+++ incubator/aurora/site/source/documentation/latest/configuration-reference.md Thu Feb 12 10:18:52 2015
@@ -29,6 +29,7 @@ Aurora + Thermos Configuration Reference
     - [UpdateConfig Objects](#updateconfig-objects)
     - [HealthCheckConfig Objects](#healthcheckconfig-objects)
     - [Announcer Objects](#announcer-objects)
+    - [Container Objects](#container)
 - [Specifying Scheduling Constraints](#specifying-scheduling-constraints)
 - [Template Namespaces](#template-namespaces)
     - [mesos Namespace](#mesos-namespace)
@@ -299,6 +300,7 @@ resources are allocated.
   ```ram```  | Integer | Bytes of RAM required by the task.
   ```disk``` | Integer | Bytes of disk required by the task.
 
+
 Job Schema
 ==========
 
@@ -322,6 +324,7 @@ Job Schema
   ```priority``` | Integer | Preemption priority to give the task (Default 0). Tasks with higher priorities may preempt tasks at lower priorities.
   ```production``` | Boolean |  Whether or not this is a production task backed by quota (Default: False). Production jobs may preempt any non-production job, and may only be preempted by production jobs in the same role and of higher priority. To run jobs at this level, the job role must have the appropriate quota.
   ```health_check_config``` | ```heath_check_config``` object | Parameters for controlling a task's health checks via HTTP. Only used if a  health port was assigned with a command line wildcard.
+  ```container``` | ```Container``` object | An optional container to run all processes inside of.
 
 ### Services
 
@@ -392,6 +395,24 @@ tasks with the same static port allocati
 External constraints such as slave attributes should be used to enforce such
 guarantees should they be needed.
 
+### Container Object
+
+*Note: The only container type currently supported is "docker".  Docker support is currently EXPERIMENTAL.*
+*Note: In order to correctly execute processes inside a job, the Docker container must have python 2.7 installed.*
+
+Describes the container the job's processes will run inside.
+
+  param          | type           | description
+  -----          | :----:         | -----------
+  ```docker```   | Docker         | A docker container to use.
+
+### Docker Object
+
+  param          | type           | description
+  -----          | :----:         | -----------
+  ```image```    | String         | The name of the docker image to execute.  If the image does not exist locally it will be pulled with ```docker pull```.
+
+
 Specifying Scheduling Constraints
 =================================
 

Modified: incubator/aurora/site/source/documentation/latest/configuration-tutorial.md
URL: http://svn.apache.org/viewvc/incubator/aurora/site/source/documentation/latest/configuration-tutorial.md?rev=1659198&r1=1659197&r2=1659198&view=diff
==============================================================================
--- incubator/aurora/site/source/documentation/latest/configuration-tutorial.md (original)
+++ incubator/aurora/site/source/documentation/latest/configuration-tutorial.md Thu Feb 12 10:18:52 2015
@@ -684,29 +684,6 @@ We'd get back:
 
     Hello Tom
 
-We can also use {{}} variables as sectional variables. Let's say we
-have:
-
-    {{#x}} Testing... {{/x}}
-
-If `x` evaluates to `True`, the text between the sectional tags is
-shown. If there is no value for `x` or it evaluates to `False`, the
-between tags text is not shown. So, at a basic level, a sectional
-variable acts as a conditional.
-
-However, if the sectional variable evaluates to a list, array, etc. it
-acts as a `foreach`. For example,
-
-    {{#x}} {{name}} {{/x}}
-
-with
-
-    { "x": [ { "name" : "tic" } { "name" : "tac" } { "name" : "toe" } ] }
-
-evaluates to
-
-    tic tac toe
-
 Every Pystachio object has an associated `.bind` method that can bind
 values to {{}} variables. Bindings are not immediately evaluated.
 Instead, they are evaluated only when the interpolated value of the

Modified: incubator/aurora/site/source/documentation/latest/contributing.md
URL: http://svn.apache.org/viewvc/incubator/aurora/site/source/documentation/latest/contributing.md?rev=1659198&r1=1659197&r2=1659198&view=diff
==============================================================================
--- incubator/aurora/site/source/documentation/latest/contributing.md (original)
+++ incubator/aurora/site/source/documentation/latest/contributing.md Thu Feb 12 10:18:52 2015
@@ -15,7 +15,7 @@ both languages.
 Find Something to Do
 --------------------
 There are issues in [Jira](https://issues.apache.org/jira/browse/AURORA) with the
-["newbie" tag](https://issues.apache.org/jira/browse/AURORA-189?jql=project%20%3D%20AURORA%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20newbie%20ORDER%20BY%20priority%20DESC)
+["newbie" label](https://issues.apache.org/jira/issues/?jql=project%20%3D%20AURORA%20AND%20labels%20%3D%20newbie%20and%20resolution%3Dunresolved)
 that are good starting places for new Aurora contributors; pick one of these and dive in! Once
 you've got a patch, the next step is to post a review.
 

Modified: incubator/aurora/site/source/documentation/latest/deploying-aurora-scheduler.md
URL: http://svn.apache.org/viewvc/incubator/aurora/site/source/documentation/latest/deploying-aurora-scheduler.md?rev=1659198&r1=1659197&r2=1659198&view=diff
==============================================================================
--- incubator/aurora/site/source/documentation/latest/deploying-aurora-scheduler.md (original)
+++ incubator/aurora/site/source/documentation/latest/deploying-aurora-scheduler.md Thu Feb 12 10:18:52 2015
@@ -12,6 +12,7 @@ machines.  This guide helps you get the
   - [Initializing the Replicated Log](#initializing-the-replicated-log)
   - [Storage Performance Considerations](#storage-performance-considerations)
   - [Network considerations](#network-considerations)
+  - [Considerations for running jobs in docker](#considerations-for-running-jobs-in-docker)
 - [Running Aurora](#running-aurora)
   - [Maintaining an Aurora Installation](#maintaining-an-aurora-installation)
   - [Monitoring](#monitoring)
@@ -33,7 +34,7 @@ machines.  This guide helps you get the
 ## Installing Aurora
 The Aurora scheduler is a standalone Java server. As part of the build process it creates a bundle
 of all its dependencies, with the notable exceptions of the JVM and libmesos. Each target server
-should have a JVM (Java 7 or higher) and libmesos (0.20.1) installed.
+should have a JVM (Java 7 or higher) and libmesos (0.21.1) installed.
 
 ### Creating the Distribution .zip File (Optional)
 To create a distribution for installation you will need build tools installed. On Ubuntu this can be
@@ -144,6 +145,23 @@ to ZooKeeper) or explicitly set in the s
     export LIBPROCESS_PORT=8083
     # ...
 
+### Considerations for running jobs in docker containers
+*Note: Docker support is currently EXPERIMENTAL.*
+
+In order for Aurora to launch jobs using docker containers, a few extra configuration options
+must be set.  The [docker containerizer](http://mesos.apache.org/documentation/latest/docker-containerizer/)
+must be enabled on the mesos slaves by launching them with the `--containerizers=docker,mesos` option.
+
+By default, aurora will configure mesos to copy the file specified in `-thermos_executor_path`
+into the container's sandbox.  If using a wrapper script to launch the thermos executor,
+specify the path to the wrapper in that arguement.  In addition, the executor pex itself must be
+included in `-thermos_executor_resources` option.  Doing so will ensure that both the wrapper script
+and executor are correctly copied into the sandbox.  In addition, ensure the wrapper script does not
+access resources outside of the sandbox, because when running inside a docker container they will not
+exist.
+
+In order to correctly execute processes inside a job, the docker container must have python 2.7 installed.
+
 ## Running Aurora
 Configure a supervisor like [Monit](http://mmonit.com/monit/) or
 [supervisord](http://supervisord.org/) to run the created `scheduler.sh` file and restart it
@@ -201,7 +219,7 @@ And this job configuration:
     )
 
 The job configuration is indicating that it should only be scheduled on slaves with the attribute
-`dedicated:dba_team/redis`.  Additionally, Aurora will prevent any tasks that do _not_ have that
+`dedicated:db_team/redis`.  Additionally, Aurora will prevent any tasks that do _not_ have that
 constraint from running on those slaves.
 
 

Modified: incubator/aurora/site/source/documentation/latest/developing-aurora-client.md
URL: http://svn.apache.org/viewvc/incubator/aurora/site/source/documentation/latest/developing-aurora-client.md?rev=1659198&r1=1659197&r2=1659198&view=diff
==============================================================================
--- incubator/aurora/site/source/documentation/latest/developing-aurora-client.md (original)
+++ incubator/aurora/site/source/documentation/latest/developing-aurora-client.md Thu Feb 12 10:18:52 2015
@@ -63,11 +63,11 @@ completes it will emit instructions for
 After following these instructions, you should now be able to run/debug tests directly from the IDE
 by right-clicking on a test (or test class) and choosing to run or debug:
 
-<a href="images/debug-client-test.png"><img src="images/debug-client-test.png" width="697" height="444" /></a>
+[![Debug Client Test](images/debug-client-test.png)](images/debug-client-test.png)
 
 If you've set a breakpoint, you can see the run will now stop and let you debug:
 
-<a href="images/debugging-client-test.png"><img src="images/debugging-client-test.png" width="697" height="444" /></a>
+[![Debugging Client Test](images/debugging-client-test.png)](images/debugging-client-test.png)
 
 ### Running/Debugging the Client
 

Modified: incubator/aurora/site/source/documentation/latest/hooks.md
URL: http://svn.apache.org/viewvc/incubator/aurora/site/source/documentation/latest/hooks.md?rev=1659198&r1=1659197&r2=1659198&view=diff
==============================================================================
--- incubator/aurora/site/source/documentation/latest/hooks.md (original)
+++ incubator/aurora/site/source/documentation/latest/hooks.md Thu Feb 12 10:18:52 2015
@@ -86,11 +86,13 @@ You can associate `pre_`, `post_`, and `
 
   Aurora Client API Method | Client API Method Argument Signature | Aurora Command Line Command
   -------------------------| ------------------------------------- | ---------------------------
-  ```cancel_update``` | ```self```, ```job_key``` | ```cancel_update```
-  ```create_job``` | ```self```, ```config``` | ```create```, <code>runtask
-  ```restart``` | ```self```, ```job_key```, ```shards```, ```update_config```, ```health_check_interval_seconds``` | ```restart```
-  ```update_job``` | ```self```, ```config```, ```health_check_interval_seconds=3```, ```shards=None``` | ```update```
-  ```kill_job``` | ```self```, ```job_key```, ```shards=None``` |  ```kill```
+  ```cancel_update``` | ```self```, ```job_key``` | ```job cancel-update```
+  ```create_job``` | ```self```, ```config``` | ```job create```, <code>runtask
+  ```restart``` | ```self```, ```job_key```, ```shards```, ```update_config```, ```health_check_interval_seconds``` | ```job restart```
+  ```update_job``` | ```self```, ```config```, ```health_check_interval_seconds=3```, ```shards=None``` | ```job update```
+  ```kill_job``` | ```self```, ```job_key```, ```shards=None``` |  ```job kill```
+  ```start_cronjob``` | ```self```, ```job_key``` | ```cron start```
+  ```start_job_update``` | ```self```, ```config```, ```instances=None``` | ```beta-update start```
 
 Some specific examples:
 
@@ -189,7 +191,7 @@ If this method returns False, the API co
 
 ## Generic Hooks
 
-There are five Aurora API Methods which any of the three hook types can attach to. Thus, there are 15 possible hook/method combinations for a single `.aurora` config file. Say that you define `pre_` and `post_` hooks for the `restart` method. That leaves 13 undefined hook/method combinations; `err_restart` and the 3 `pre_`, `post_`, and `err_` hooks for each of the other 4 hookable methods. You can define what happens when any of these otherwise undefined 13 hooks execute via a generic hook, whose signature is:
+There are seven Aurora API Methods which any of the three hook types can attach to. Thus, there are 21 possible hook/method combinations for a single `.aurora` config file. Say that you define `pre_` and `post_` hooks for the `restart` method. That leaves 19 undefined hook/method combinations; `err_restart` and the 3 `pre_`, `post_`, and `err_` hooks for each of the other 6 hookable methods. You can define what happens when any of these otherwise undefined 19 hooks execute via a generic hook, whose signature is:
 
 ```python
 generic_hook(self, hook_config, event, method_name, result_or_err, args*, kw**)

Modified: incubator/aurora/site/source/documentation/latest/vagrant.md
URL: http://svn.apache.org/viewvc/incubator/aurora/site/source/documentation/latest/vagrant.md?rev=1659198&r1=1659197&r2=1659198&view=diff
==============================================================================
--- incubator/aurora/site/source/documentation/latest/vagrant.md (original)
+++ incubator/aurora/site/source/documentation/latest/vagrant.md Thu Feb 12 10:18:52 2015
@@ -1,49 +1,134 @@
 Getting Started
 ===============
-To replicate a real cluster environment as closely as possible, we use
-[Vagrant](http://www.vagrantup.com/) to launch a complete Aurora cluster in a virtual machine.
 
-Prerequisites
--------------
-  * [VirtualBox](https://www.virtualbox.org/)
-  * [Vagrant](http://www.vagrantup.com/)
-  * A clone of the Aurora repository, or source distribution.
+This document shows you how to configure a complete cluster using a virtual machine. This setup
+replicates a real cluster in your development machine as closely as possible. After you complete
+the steps outlined here, you will be ready to create and run your first Aurora job.
 
-You can start a local cluster by running:
+The following sections describe these steps in detail:
 
-    vagrant up
+1. [Overview](#user-content-overview)
+1. [Install VirtualBox and Vagrant](#user-content-install-virtualbox-and-vagrant)
+1. [Clone the Aurora repository](#user-content-clone-the-aurora-repository)
+1. [Start the local cluster](#user-content-start-the-local-cluster)
+1. [Log onto the VM](#user-content-log-onto-the-vm)
+1. [Run your first job](#user-content-run-your-first-job)
+1. [Rebuild components](#user-content-rebuild-components)
+1. [Shut down or delete your local cluster](#user-content-shut-down-or-delete-your-local-cluster)
+1. [Troubleshooting](#user-content-troubleshooting)
 
-Once started, several services should be running:
 
-  * scheduler is listening on http://192.168.33.7:8081
-  * observer is listening on http://192.168.33.7:1338
-  * master is listening on http://192.168.33.7:5050
-  * slave is listening on http://192.168.33.7:5051
+Overview
+--------
 
-You can SSH into the machine with `vagrant ssh` and execute aurora client commands using the
-`aurora` command.  A pre-installed `clusters.json` file refers to your local cluster as
-`devcluster`, which you will use in client commands.
+The Aurora distribution includes a set of scripts that enable you to create a local cluster in
+your development machine. These scripts use [Vagrant](https://www.vagrantup.com/) and
+[VirtualBox](https://www.virtualbox.org/) to run and configure a virtual machine. Once the
+virtual machine is running, the scripts install and initialize Aurora and any required components
+to create the local cluster.
 
-Deleting your local cluster
-===========================
-Once you are finished with your local cluster, or if you would otherwise like to start from scratch,
-you can use the command `vagrant destroy` to turn off and delete the virtual file system.
+
+Install VirtualBox and Vagrant
+------------------------------
+
+First, download and install [VirtualBox](https://www.virtualbox.org/) on your development machine.
+
+Then download and install [Vagrant](https://www.vagrantup.com/). To verify that the installation
+was successful, open a terminal window and type the `vagrant` command. You should see a list of
+common commands for this tool.
+
+
+Clone the Aurora repository
+---------------------------
+
+To obtain the Aurora source distribution, clone its Git repository using the following command:
+
+     git clone http://git.apache.org/incubator-aurora.git
+
+
+Start the local cluster
+-----------------------
+
+Now change into the `incubator-aurora/` directory, which contains the Aurora source code and
+other scripts and tools:
+
+     cd incubator-aurora/
+
+To start the local cluster, type the following command:
+
+     vagrant up
+
+This command uses the configuration scripts in the Aurora distribution to:
+
+* Download a Linux system image.
+* Start a virtual machine (VM) and configure it.
+* Install the required build tools on the VM.
+* Install Aurora's requirements (like [Mesos](http://mesos.apache.org/) and
+[Zookeeper](http://zookeeper.apache.org/)) on the VM.
+* Build and install Aurora from source on the VM.
+* Start Aurora's services on the VM.
+
+This process takes several minutes to complete.
+
+To verify that Aurora is running on the cluster, visit the following URLs:
+
+* Scheduler - http://192.168.33.7:8081
+* Observer - http://192.168.33.7:1338
+* Mesos Master - http://192.168.33.7:5050
+* Mesos Slave - http://192.168.33.7:5051
 
 
-Rebuilding components
-=====================
+Log onto the VM
+---------------
+
+To SSH into the VM, run the following command in your development machine:
+
+     vagrant ssh
+
+To verify that Aurora is installed in the VM, type the `aurora` command. You should see a list
+of arguments and possible commands.
+
+The `/vagrant` directory on the VM is mapped to the `incubator-aurora/` local directory
+from which you started the cluster. You can edit files inside this directory in your development
+machine and access them from the VM under `/vagrant`.
+
+A pre-installed `clusters.json` file refers to your local cluster as `devcluster`, which you
+will use in client commands.
+
+
+Run your first job
+------------------
+
+Now that your cluster is up and running, you are ready to define and run your first job in Aurora.
+For more information, see the [Aurora Tutorial](/documentation/latest/tutorial/).
+
+
+Rebuild components
+------------------
+
 If you are changing Aurora code and would like to rebuild a component, you can use the `aurorabuild`
-command on your vagrant machine to build and restart a component.  This is considerably faster than
-destroying and rebuilding your VM.
+command on the VM to build and restart a component.  This is considerably faster than destroying
+and rebuilding your VM.
 
-`aurorabuild` accepts a list of components to build and update.  You may invoke the command with
-no arguments to get a list of supported components.
+`aurorabuild` accepts a list of components to build and update. To get a list of supported
+components, invoke the `aurorabuild` command with no arguments:
 
      vagrant ssh -c 'aurorabuild client'
 
 
+Shut down or delete your local cluster
+--------------------------------------
+
+To shut down your local cluster, run the `vagrant halt` command in your development machine. To
+start it again, run the `vagrant up` command.
+
+Once you are finished with your local cluster, or if you would otherwise like to start from scratch,
+you can use the command `vagrant destroy` to turn off and delete the virtual file system.
+
+
 Troubleshooting
-===============
+---------------
+
 Most of the vagrant related problems can be fixed by the following steps:
 
 * Destroying the vagrant environment with `vagrant destroy`



Mime
View raw message