mesos-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dles...@apache.org
Subject [2/2] git commit: Renames filenames of documentation to be lowercase and updates some documentation templates for consistency.
Date Tue, 29 Oct 2013 03:56:03 GMT
Renames filenames of documentation to be lowercase and updates some documentation templates for consistency.


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/36a78668
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/36a78668
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/36a78668

Branch: refs/heads/master
Commit: 36a7866807023ad7b51201bfcd6dca1831f0e704
Parents: 194193e
Author: Dave Lester <dlester@twitter.com>
Authored: Mon Oct 28 20:55:13 2013 -0700
Committer: Dave Lester <dlester@twitter.com>
Committed: Mon Oct 28 20:55:13 2013 -0700

----------------------------------------------------------------------
 docs/Allocation-module.md                       | 132 ------------
 docs/App-Framework-Development-Guide.md         | 208 -------------------
 docs/Configuration.md                           |  88 --------
 docs/Deploy-Scripts.md                          |  16 --
 docs/EC2-Scripts.md                             |  59 ------
 docs/Getting-Started.md                         | 104 ----------
 docs/High-Availability.md                       |  50 -----
 docs/Home.md                                    |  44 ----
 docs/Logging-and-Debugging.md                   |  11 -
 docs/Mesos-Architecture.md                      |  32 ---
 docs/Mesos-Code-Internals.md                    |  41 ----
 docs/Mesos-Presentations.md                     |  46 ----
 docs/Mesos-Roadmap.md                           |   7 -
 docs/Mesos-c++-style-guide.md                   |  72 -------
 docs/Mesos-developers-guide.md                  |  72 -------
 docs/Powered-by-Mesos.md                        |  29 ---
 docs/Running-a-web-application-farm-on-mesos.md |  25 ---
 docs/Running-torque-or-mpi-on-mesos.md          |   9 -
 docs/Slave-Recovery.md                          |  66 ------
 docs/Upgrades.md                                |  58 ------
 docs/Using-the-mesos-submit-tool.md             |   9 -
 docs/allocation-module.md                       | 132 ++++++++++++
 docs/app-framework-development-guide.md         | 208 +++++++++++++++++++
 docs/configuration.md                           |  88 ++++++++
 docs/deploy-scripts.md                          |  16 ++
 docs/ec2-scripts.md                             |  59 ++++++
 docs/getting-started.md                         | 104 ++++++++++
 docs/high-availability.md                       |  50 +++++
 docs/home.md                                    |  43 ++++
 docs/logging-and-debugging.md                   |  11 +
 docs/mesos-architecture.md                      |  32 +++
 docs/mesos-c++-style-guide.md                   |  72 +++++++
 docs/mesos-code-internals.md                    |  41 ++++
 docs/mesos-developers-guide.md                  |  72 +++++++
 docs/mesos-presentations.md                     |  46 ++++
 docs/mesos-roadmap.md                           |   9 +
 docs/powered-by-mesos.md                        |  29 +++
 docs/running-a-web-application-farm-on-mesos.md |  25 +++
 docs/running-torque-or-mpi-on-mesos.md          |   9 +
 docs/slave-recovery.md                          |  66 ++++++
 docs/upgrades.md                                |  58 ++++++
 docs/using-the-mesos-submit-tool.md             |   9 +
 42 files changed, 1179 insertions(+), 1178 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Allocation-module.md
----------------------------------------------------------------------
diff --git a/docs/Allocation-module.md b/docs/Allocation-module.md
deleted file mode 100644
index b308a23..0000000
--- a/docs/Allocation-module.md
+++ /dev/null
@@ -1,132 +0,0 @@
----
-layout: documentation
----
-
-# Mesos Allocation Module
-
-The logic that the Mesos master uses to determine which frameworks to make offer resource offers to is encapsulated in the Master's _allocation module_.  The allocation module is a pluggable component that organizations can use to implement their own sharing policy, e.g. fair-sharing, Dominant Resource Fairness (see [the DRF paper](http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-55.pdf)), priority, etc.
-
-## Allocation Module API
-
-Mesos is implemented in C++, so allocation modules are implemented in C++, and inherit the @AllocatorProcess@ class defined in @MESOS_HOME/src/master/allocator.hpp@. As of the time of this writing (5/29/13), the API for allocation modules is as follows:
-
-```
-  virtual ~AllocatorProcess() {}
-
-  virtual void initialize(
-      const Flags& flags,
-      const process::PID<Master>& master,
-      const hashmap<std::string, RoleInfo>& roles) = 0;
-
-  virtual void frameworkAdded(
-      const FrameworkID& frameworkId,
-      const FrameworkInfo& frameworkInfo,
-      const Resources& used) = 0;
-
-  virtual void frameworkRemoved(
-      const FrameworkID& frameworkId) = 0;
-
-  virtual void frameworkActivated(
-      const FrameworkID& frameworkId,
-      const FrameworkInfo& frameworkInfo) = 0;
-
-  virtual void frameworkDeactivated(
-      const FrameworkID& frameworkId) = 0;
-
-  virtual void slaveAdded(
-      const SlaveID& slaveId,
-      const SlaveInfo& slaveInfo,
-      const hashmap<FrameworkID, Resources>& used) = 0;
-
-  virtual void slaveRemoved(
-      const SlaveID& slaveId) = 0;
-
-  virtual void updateWhitelist(
-      const Option<hashset<std::string> >& whitelist) = 0;
-
-  virtual void resourcesRequested(
-      const FrameworkID& frameworkId,
-      const std::vector<Request>& requests) = 0;
-
-  // Whenever resources offered to a framework go unused (e.g.,
-  // refused) the master invokes this callback.
-  virtual void resourcesUnused(
-      const FrameworkID& frameworkId,
-      const SlaveID& slaveId,
-      const Resources& resources,
-      const Option<Filters>& filters) = 0;
-
-  // Whenever resources are "recovered" in the cluster (e.g., a task
-  // finishes, an offer is removed because a framework has failed or
-  // is failing over) the master invokes this callback.
-  virtual void resourcesRecovered(
-      const FrameworkID& frameworkId,
-      const SlaveID& slaveId,
-      const Resources& resources) = 0;
-
-  // Whenever a framework that has filtered resources wants to revive
-  // offers for those resources the master invokes this callback.
-  virtual void offersRevived(
-      const FrameworkID& frameworkId) = 0;
-```
-
-The default allocation module is the HierarchicalAllocatorProcess, which can be found in @MESOS_HOME/src/master/hierarchical_allocator_process.hpp@. You can reference this as a starting place if you choose to write your own allocation module.
-
-## Sorter API
-
-Additionally, the hierarchical allocator module can be extended without the need to reimplement the entirety of the allocation logic through the use of the @Sorter@ abstraction.
-
-Sorters define the order that roles or frameworks should be offered resources in by taking "client" objects and some information about those clients and returning an ordered list of clients.
-
-Sorters are implemented in C++ and inherit the @Sorter@ class defined in @MESOS_HOME/src/master/sorter.hpp@. As of the time of this writing, the API for Sorters is as follows:
-
-```
-  virtual ~Sorter() {}
-
-  // Adds a client to allocate resources to. A client
-  // may be a user or a framework.
-  virtual void add(const std::string& client, double weight = 1) = 0;
-
-  // Removes a client.
-  virtual void remove(const std::string& client) = 0;
-
-  // Readds a client to the sort after deactivate.
-  virtual void activate(const std::string& client) = 0;
-
-  // Removes a client from the sort, so it won't get allocated to.
-  virtual void deactivate(const std::string& client) = 0;
-
-  // Specify that resources have been allocated to the given client.
-  virtual void allocated(const std::string& client,
-                         const Resources& resources) = 0;
-
-  // Specify that resources have been unallocated from the given client.
-  virtual void unallocated(const std::string& client,
-                           const Resources& resources) = 0;
-
-  // Returns the resources that have been allocated to this client.
-  virtual Resources allocation(const std::string& client) = 0;
-
-  // Add resources to the total pool of resources this
-  // Sorter should consider.
-  virtual void add(const Resources& resources) = 0;
-
-  // Remove resources from the total pool.
-  virtual void remove(const Resources& resources) = 0;
-
-  // Returns a list of all clients, in the order that they
-  // should be allocated to, according to this Sorter's policy.
-  virtual std::list<std::string> sort() = 0;
-
-  // Returns true if this Sorter contains the specified client,
-  // either active or deactivated.
-  virtual bool contains(const std::string& client) = 0;
-
-  // Returns the number of clients this Sorter contains,
-  // either active or deactivated.
-  virtual int count() = 0;
-```
-
-The default @Sorter@ is the DRFSorter, which implements fair sharing and can be found at @MESOS_HOME/src/master/drf_sorter.hpp@.
-
-For DRF, if weights are specified in Sorter::add, a client's share will be divided by the weight, creating a form of priority. For example, a role that has a weight of 2 will be offered twice as many resources as a role with weight 1.
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/App-Framework-Development-Guide.md
----------------------------------------------------------------------
diff --git a/docs/App-Framework-Development-Guide.md b/docs/App-Framework-Development-Guide.md
deleted file mode 100644
index 6d75e4a..0000000
--- a/docs/App-Framework-Development-Guide.md
+++ /dev/null
@@ -1,208 +0,0 @@
----
-layout: documentation
----
-
-# Framework Development Guide
-
-In this document we refer to Mesos applications as "frameworks".
-
-See one of the example framework schedulers in @MESOS_HOME/src/examples/@ to get an idea of what a Mesos framework scheduler and executor in the language of your choice looks like.
-
-## Create your Framework Scheduler
-
-You can write a framework scheduler in C, C++, Java/Scala, or Python. Your framework scheduler should inherit from the @Scheduler@ class (see API below). Your scheduler should create a SchedulerDriver (which will mediate communication between your scheduler and the Mesos master) and then call @SchedulerDriver.run()@
-
-### Scheduler API
-
-Declared in @MESOS_HOME/include/mesos/scheduler.hpp@
-
-```
-  /**
-   * Empty virtual destructor (necessary to instantiate subclasses).
-   */
-  virtual ~Scheduler() {}
-
-  /**
-   * Invoked when the scheduler successfully registers with a Mesos
-   * master. A unique ID (generated by the master) used for
-   * distinguishing this framework from others and MasterInfo
-   * with the ip and port of the current master are provided as arguments.
-   */
-  virtual void registered(SchedulerDriver* driver,
-                          const FrameworkID& frameworkId,
-                          const MasterInfo& masterInfo) = 0;
-
-  /**
-   * Invoked when the scheduler re-registers with a newly elected Mesos master.
-   * This is only called when the scheduler has previously been registered.
-   * MasterInfo containing the updated information about the elected master
-   * is provided as an argument.
-   */
-  virtual void reregistered(SchedulerDriver* driver,
-                            const MasterInfo& masterInfo) = 0;
-
-  /**
-   * Invoked when the scheduler becomes "disconnected" from the master
-   * (e.g., the master fails and another is taking over).
-   */
-  virtual void disconnected(SchedulerDriver* driver) = 0;
-
-  /**
-   * Invoked when resources have been offered to this framework. A
-   * single offer will only contain resources from a single slave.
-   * Resources associated with an offer will not be re-offered to
-   * _this_ framework until either (a) this framework has rejected
-   * those resources (see SchedulerDriver::launchTasks) or (b) those
-   * resources have been rescinded (see Scheduler::offerRescinded).
-   * Note that resources may be concurrently offered to more than one
-   * framework at a time (depending on the allocator being used). In
-   * that case, the first framework to launch tasks using those
-   * resources will be able to use them while the other frameworks
-   * will have those resources rescinded (or if a framework has
-   * already launched tasks with those resources then those tasks will
-   * fail with a TASK_LOST status and a message saying as much).
-   */
-  virtual void resourceOffers(SchedulerDriver* driver,
-                              const std::vector<Offer>& offers) = 0;
-
-  /**
-   * Invoked when an offer is no longer valid (e.g., the slave was
-   * lost or another framework used resources in the offer). If for
-   * whatever reason an offer is never rescinded (e.g., dropped
-   * message, failing over framework, etc.), a framwork that attempts
-   * to launch tasks using an invalid offer will receive TASK_LOST
-   * status updats for those tasks (see Scheduler::resourceOffers).
-   */
-  virtual void offerRescinded(SchedulerDriver* driver,
-                              const OfferID& offerId) = 0;
-
-  /**
-   * Invoked when the status of a task has changed (e.g., a slave is
-   * lost and so the task is lost, a task finishes and an executor
-   * sends a status update saying so, etc). Note that returning from
-   * this callback _acknowledges_ receipt of this status update! If
-   * for whatever reason the scheduler aborts during this callback (or
-   * the process exits) another status update will be delivered (note,
-   * however, that this is currently not true if the slave sending the
-   * status update is lost/fails during that time).
-   */
-  virtual void statusUpdate(SchedulerDriver* driver,
-                            const TaskStatus& status) = 0;
-
-  /**
-   * Invoked when an executor sends a message. These messages are best
-   * effort; do not expect a framework message to be retransmitted in
-   * any reliable fashion.
-   */
-  virtual void frameworkMessage(SchedulerDriver* driver,
-                                const ExecutorID& executorId,
-                                const SlaveID& slaveId,
-                                const std::string& data) = 0;
-
-  /**
-   * Invoked when a slave has been determined unreachable (e.g.,
-   * machine failure, network partition). Most frameworks will need to
-   * reschedule any tasks launched on this slave on a new slave.
-   */
-  virtual void slaveLost(SchedulerDriver* driver,
-                         const SlaveID& slaveId) = 0;
-
-  /**
-   * Invoked when an executor has exited/terminated. Note that any
-   * tasks running will have TASK_LOST status updates automagically
-   * generated.
-   */
-  virtual void executorLost(SchedulerDriver* driver,
-                            const ExecutorID& executorId,
-                            const SlaveID& slaveId,
-                            int status) = 0;
-
-  /**
-   * Invoked when there is an unrecoverable error in the scheduler or
-   * scheduler driver. The driver will be aborted BEFORE invoking this
-   * callback.
-   */
-  virtual void error(SchedulerDriver* driver, const std::string& message) = 0;
-```
-
-## Create your Framework Executor
-
-Your framework executor must inherit from the Executor class. It must override the launchTask() method. You can use the $MESOS_HOME environment variable inside of your executor to determine where mesos is running from.
-
-### Executor API
-
-Declared in @MESOS_HOME/include/mesos/executor.hpp@
-
-```
-  /**
-   * Invoked once the executor driver has been able to successfully
-   * connect with Mesos. In particular, a scheduler can pass some
-   * data to it's executors through the FrameworkInfo.ExecutorInfo's
-   * data field.
-   */
-  virtual void registered(ExecutorDriver* driver,
-                          const ExecutorInfo& executorInfo,
-                          const FrameworkInfo& frameworkInfo,
-                          const SlaveInfo& slaveInfo) = 0;
-
-  /**
-   * Invoked when the executor re-registers with a restarted slave.
-   */
-  virtual void reregistered(ExecutorDriver* driver,
-                            const SlaveInfo& slaveInfo) = 0;
-
-  /**
-   * Invoked when the executor becomes "disconnected" from the slave
-   * (e.g., the slave is being restarted due to an upgrade).
-   */
-  virtual void disconnected(ExecutorDriver* driver) = 0;
-
-  /**
-   * Invoked when a task has been launched on this executor (initiated
-   * via Scheduler::launchTasks). Note that this task can be realized
-   * with a thread, a process, or some simple computation, however, no
-   * other callbacks will be invoked on this executor until this
-   * callback has returned.
-   */
-  virtual void launchTask(ExecutorDriver* driver,
-                          const TaskInfo& task) = 0;
-
-  /**
-   * Invoked when a task running within this executor has been killed
-   * (via SchedulerDriver::killTask). Note that no status update will
-   * be sent on behalf of the executor, the executor is responsible
-   * for creating a new TaskStatus (i.e., with TASK_KILLED) and
-   * invoking ExecutorDriver::sendStatusUpdate.
-   */
-  virtual void killTask(ExecutorDriver* driver, const TaskID& taskId) = 0;
-
-  /**
-   * Invoked when a framework message has arrived for this
-   * executor. These messages are best effort; do not expect a
-   * framework message to be retransmitted in any reliable fashion.
-   */
-    virtual void frameworkMessage(ExecutorDriver* driver,
-                                  const std::string& data) = 0;
-
-  /**
-   * Invoked when the executor should terminate all of it's currently
-   * running tasks. Note that after a Mesos has determined that an
-   * executor has terminated any tasks that the executor did not send
-   * terminal status updates for (e.g., TASK_KILLED, TASK_FINISHED,
-   * TASK_FAILED, etc) a TASK_LOST status update will be created.
-   */
-  virtual void shutdown(ExecutorDriver* driver) = 0;
-
-  /**
-   * Invoked when a fatal error has occured with the executor and/or
-   * executor driver. The driver will be aborted BEFORE invoking this
-   * callback.
-   */
-  virtual void error(ExecutorDriver* driver, const std::string& message) = 0;
-```
-
-## Install your Framework
-
-You need to put your framework somewhere that all slaves on the cluster can get it from. If you are running HDFS, you can put your executor into HDFS. Then, you tell Mesos where it is via the @ExecutorInfo@ parameter of @MesosSchedulerDriver@'s constructor (e.g. see src/examples/java/TestFramework.java for an example of this). ExecutorInfo is a a Protocol Buffer Message class (defined in @include/mesos/mesos.proto@), and you set its uri field to something like "HDFS://path/to/executor/". Also, you can pass the @frameworks_home@ configuration option (defaults to: @MESOS_HOME/frameworks@) to your @mesos-slave@ daemons when you launch them to specify where all of your framework executors are stored (e.g. on an NFS mount that is available to all slaves), then set @ExecutorInfo@ to be a relative path, and the slave will prepend the value of frameworks_home to the relative path provided.
-
-Once you are sure that your executors are available to the mesos-slaves, you should be able to run your scheduler, which will register with the Mesos master, and start receiving resource offers!
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Configuration.md
----------------------------------------------------------------------
diff --git a/docs/Configuration.md b/docs/Configuration.md
deleted file mode 100644
index 423c53a..0000000
--- a/docs/Configuration.md
+++ /dev/null
@@ -1,88 +0,0 @@
----
-layout: documentation
----
-
-# Mesos Configuration
-
-The Mesos master and slave can take a variety of configuration options through command-line arguments, or environment variables. A list of the available options can be seen by running `mesos-master --help` or `mesos-slave --help`. Each option can be set in two ways:
-
-* By passing it to the binary using `--option_name=value`.
-* By setting the environment variable `MESOS_OPTION_NAME` (the option name with a `MESOS_` prefix added to it).
-
-Configuration values are searched for first in the environment, then on the command-line.
-
-**Important Options**
-
-If you have special compilation requirements, please refer to `./configure --help` when configuring Mesos. Additionally, the documentation lists only a subset of the options. A definitive source for which flags your version of Mesos supports can be found by running the binary with the flag `--help`, for example `bin/mesos-master --help`.
-
-## Master and Slave Options
-
-*These options can be supplied to both masters and slaves.*
-
-* `log_dir` : Directory for log files. If unspecified, nothing is written to disk.
-
-* `quiet` : Disable logging to stderr. [Default: false]
-
-* `ip` : IP address to listen on. [Optional]
-
-* `port` : Port to listen on. [Default for master: 5050, Default for slave:5051]
-
-## Master Options
-
-* `zk` : ZooKeeper URL (used for leader election amongst multiple masters). May be one of:
-	* zk://host1:port1,host2:port2,.../path
-	* zk://username:password@host1:port1,host2:port2,.../path
-	* file://path/to/file (where file contains one of the above)
-
-* `roles` : A comma seperated list of the allocation roles
-            that frameworks in this cluster may belong to.
-
-* `weights` : A comma seperated list of role/weight pairs
-              of the form 'role=weight,role=weight'. Weights
-              are used to indicate forms of priority.
-
-## Slave Options
-
-* `resources` : Total consumable resources per slave, in
-                  the form 'name(role):value;name(role):value...'.
-	* NOTE: '(role)' is optional.
-	* Ex: "cpus(role2):2;mem(role2):1024;cpus:1;mem:1024;disk:0"
-
-* `attributes` : Attributes for the machine. [Optional]
-	* These are free-form in the same style as resources, ex: "rack:abc;kernel:2.6.44".
-	* This information is provided to frameworks.
-
-* `work_dir` : Directory for the executor work directories. [Default: /tmp/mesos]
-	* `isolation` : Isolation mechanism, one of: "process", "cgroups". [Default: process]
-	* `cgroups_enable_cfs` : If using cgroups isolation, this enables hard limits on CPU resources.
-
-* `master` : May be one of:
-	* zk://host1:port1,host2:port2,.../path
-	* zk://username:password@host1:port1,host2:port2,.../path\n
-	* file://path/to/file (where file contains one of the above)
-
-* `default_role` : Any resources in the --resources flag that
-                   omit a role, as well as any resources that
-                   are not present in --resources but that are
-                   automatically detected, will be assigned to
-                   this role. [Default: *]
-
-* `checkpoint` :  Whether to checkpoint slave and frameworks information
-                  to disk.
-	* This enables a restarted slave to recover status updates and reconnect
-    with (--recover=reconnect) or kill (--recover=kill) old executors [Default: false]
-
-* `strict` : Whether to do recovery in strict mode [Default: true].
-	* If strict=true, any and all recovery errors are considered fatal.
-	* If strict=false, any errors (e.g., corruption in checkpointed data) during recovery are
-    ignored and as much state as possible is recovered.
-
-## Mesos Configuration Options
-
-The configure script itself accepts the following arguments to enable various options:
-
-* `--with-python-headers=DIR`: Find Python header files in `DIR` (to turn on Python support). Recommended.
-* `--with-webui`: Enable the Mesos web UI (which requires Python 2.6). Recommended.
-* `--with-java-home=DIR`: Enable Java application/framework support with a given installation of Java. Required for Hadoop and Spark.
-* `--with-java-headers=DIR`: Find Java header files (necessary for newer versions of OS X Snow Leopard).
-* `--with-included-zookeeper` or `--with-zookeeper=DIR`: Enable master fault-tolerance using an existing ZooKeeper installation or the version of ZooKeeper bundled with Mesos. For details, see [High Availability](High-Availability.md).
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Deploy-Scripts.md
----------------------------------------------------------------------
diff --git a/docs/Deploy-Scripts.md b/docs/Deploy-Scripts.md
deleted file mode 100644
index cb04d5f..0000000
--- a/docs/Deploy-Scripts.md
+++ /dev/null
@@ -1,16 +0,0 @@
----
-layout: documentation
----
-
-# Deploy Scripts
-
-Mesos includes a set of scripts in @MESOS_HOME/deploy@ that can be used to deploy it on a cluster. To use these scripts, you need to create two configuration files: @MESOS_HOME/conf/masters@, which should list the hostname(s) of the node(s) you want to be your masters (one per line), and @MESOS_HOME/conf/slaves@, which should contain a list of hostnames for your slaves. You can then start a cluster with @deploy/start-mesos@ and stop it with @deploy/stop-mesos@.
-
-It is also possible to set environment variables, ulimits, etc that will affect the master and slave. by editing @MESOS_HOME/deploy/mesos-env.sh@. One particularly useful setting is @LIBPROCESS_IP@, which tells the master and slave binaries which IP address to bind to; in some installations, the default interface that the hostname resolves to is not the machine's external IP address, so you can set the right IP through this variable.
-
-Finally, the deploy scripts do not use ZooKeeper by default. If you want to use ZooKeeper (for multiple masters), you can do so by either editing @deploy/mesos-env.sh@ to set @MESOS_URL@ to a @zoo://@ or @zoofile://@ URL, or by editing @conf/mesos.conf@ to set the @url@ configuration parameter. Please see the "High Availability" documentation for details.
-
-## Notes
-
-* The deploy scripts assume that Mesos is located in the same directory on all nodes.
-* If you want to enable multiple Unix users to submit to the same cluster, you need to run the Mesos slaves as root (or possibly set the right attributes on the @mesos-slave@ binary). Otherwise, they will fail to @setuid@.
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/EC2-Scripts.md
----------------------------------------------------------------------
diff --git a/docs/EC2-Scripts.md b/docs/EC2-Scripts.md
deleted file mode 100644
index a09c73c..0000000
--- a/docs/EC2-Scripts.md
+++ /dev/null
@@ -1,59 +0,0 @@
----
-layout: documentation
----
-
-# EC2 Scripts
-
-The @mesos-ec2@ script located in the Mesos's @ec2@ directory allows you to launch, manage and shut down Mesos clusters on Amazon EC2. You don't need to build Mesos to use this script -- you just need Python 2.6+ installed.
-
-@mesos-ec2@ is designed to manage multiple named clusters. You can launch a new cluster (telling the script its size and giving it a name), shutdown an existing cluster, or log into a cluster. Each cluster is identified by placing its machines into EC2 security groups whose names are derived from the name of the cluster. For example, a cluster named @test@ will contain a master node in a security group called @test-master@, and a number of slave nodes in a security group called @test-slaves@. The @mesos-ec2@ script will create these security groups for you based on the cluster name you request. You can also use them to identify machines belonging to each cluster in the EC2 Console or ElasticFox.
-
-This guide describes how to get set up to run clusters, how to launch clusters, how to run jobs on them, and how to shut them down.
-
-## Before You Start
-
-* Create an Amazon EC2 key pair for yourself. This can be done by logging into your Amazon Web Services account through the "AWS console":http://aws.amazon.com/console/, clicking Key Pairs on the left sidebar, and creating and downloading a key. Make sure that you set the permissions for the private key file to @600@ (i.e. only you can read and write it) so that @ssh@ will work.
-* Whenever you want to use the @mesos-ec2@ script, set the environment variables @AWS_ACCESS_KEY_ID@ and @AWS_SECRET_ACCESS_KEY@ to your Amazon EC2 access key ID and secret access key. These can be obtained from the "AWS homepage":http://aws.amazon.com/ by clicking Account > Security Credentials > Access Credentials.
-
-## Launching a Cluster
-
-* Go into the @ec2@ directory in the release of Mesos you downloaded.
-* Run @./mesos-ec2 -k <keypair> -i <key-file> -s <num-slaves> launch <cluster-name>@, where @<keypair>@ is the name of your EC2 key pair (that you gave it when you created it), @<key-file>@ is the private key file for your key pair, @<num-slaves>@ is the number of slave nodes to launch (try 1 at first), and @<cluster-name>@ is the name to give to your cluster.
-* After everything launches, check that Mesos is up and sees all the slaves by going to the Mesos Web UI link printed at the end of the script (@http://<master-hostname>:8080@).
-
-You can also run @./mesos-ec2 --help@ to see more usage options. The following options are worth pointing out:
-
-* @--instance-type=<INSTANCE_TYPE>@ can be used to specify an EC2 instance type to use. For now, the script only supports 64-bit instance types, and the default type is @m1.large@ (which has 2 cores and 7.5 GB RAM). Refer to the Amazon pages about "EC2 instance types":http://aws.amazon.com/ec2/instance-types and "EC2 pricing":http://aws.amazon.com/ec2/#pricing for information about other instance types. 
-* @--zone=<EC2_ZONE>@ can be used to specify an EC2 availability zone to launch instances in. Sometimes, you will get an error because there is not enough capacity in one zone, and you should try to launch in another. This happens mostly with the @m1.large@ instance types; extra-large (both @m1.xlarge@ and @c1.xlarge@) instances tend to be more available.
-* @--download=git@ will tell the instances to download the latest release of Mesos from the github git repository.
-* @--ft=<NUM_MASTERS>@ can be used to run Mesos in fault tolerant (FT) mode by specifying @NUM_MASTERS@ > 1.
-* If one of your launches fails due to e.g. not having the right permissions on your private key file, you can run @launch@ with the @--resume@ option to restart the setup process on an existing cluster.
-
-## Running Jobs
-
-* Go into the @ec2@ directory in the release of Mesos you downloaded.
-* Run @./mesos-ec2 -k <keypair> -i <key-file> login <cluster-name>@ to SSH into the cluster, where @<keypair>@ and @<key-file>@ are as above. (This is just for convenience; you could also use Elasticfox or the EC2 console.)
-* Copy your code to all the nodes. To do this, you can use the provided script @~/mesos-ec2/copy-dir@, which, given a directory path, RSYNCs it to the same location on all the slaves.
-* If your job needs to access large datasets, the fastest way to do that is to load them from Amazon S3 or an Amazon EBS device into an instance of the Hadoop Distributed File System (HDFS) on your nodes. The @mesos-ec2@ script already sets up a HDFS instance for you. It's installed in @/root/ephemeral-hdfs@, and can be accessed using the @bin/hadoop@ script in that directory. Note that the data in this HDFS goes away when you stop and restart a machine.
-* There is also a _persistent HDFS_ instance in @/root/presistent-hdfs@ that will keep data across cluster restarts. Typically each node has relatively little space of persistent data (about 3 GB), but you can use the @--ebs-vol-size@ option to @mesos-ec2@ to attach a persistent EBS volume to each node for storing the persistent HDFS.
-
-If you get an "Executor on slave X disconnected" error when running your framework, you probably haven't copied your code the slaves. Use the @~/mesos-ec2/copy-dir@ script to do that. If you keep getting the error, though, look at the slave's logs for that framework using the Mesos web UI. Please see "logging and debugging":Logging-and-Debugging.textile for details.
-
-## Terminating a Cluster
-
-_*Note that there is no way to recover data on EC2 nodes after shutting them down! Make sure you have copied everything important off the nodes before stopping them.*_
-
-* Go into the @ec2@ directory in the release of Mesos you downloaded.
-* Run @./mesos-ec2 destroy <cluster-name>@.
-
-## Pausing and Restarting EBS-Backed Clusters
-
-The @mesos-ec2@ script also supports pausing a cluster if you are using EBS-backed virtual machines (which all of our machine images are by default). In this case, the VMs are stopped but not terminated, so they _*lose all data on ephemeral disks (/mnt, ephemeral-hdfs)*_ but keep the data in their root partitions and their @persistent-hdfs@. Stopped machines will not cost you any EC2 cycles, but _*will*_ continue to cost money for EBS storage.
-
-* To stop one of your clusters, go into the @ec2@ directory and run @./mesos-ec2 stop <cluster-name>@.
-* To restart it later, run @./mesos-ec2 -i <key-file> start <cluster-name>@.
-* To ultimately destroy the cluster and stop consuming EBS space, run @./mesos-ec2 destroy <cluster-name>@ as described in the previous section.
-
-## Limitations
-
-* The @mesos-ec2@ script currently does not use the "deploy scripts":Deploy-Scripts.textile included with Mesos to manage its clusters. This will likely be fixed in the future.
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Getting-Started.md
----------------------------------------------------------------------
diff --git a/docs/Getting-Started.md b/docs/Getting-Started.md
deleted file mode 100644
index 3986dab..0000000
--- a/docs/Getting-Started.md
+++ /dev/null
@@ -1,104 +0,0 @@
----
-layout: gettingstarted
----
-
-# Getting Started
-
-## Downloading Mesos
-
-There are different ways you can get Mesos:
-
-1. Download the latest stable release from [Apache](http://mesos.apache.org/downloads/) (***Recommended***)
-
-        $ wget http://www.apache.org/dist/mesos/0.14.0/mesos-0.14.0.tar.gz
-        $ tar -zxf mesos-0.14.0.tar.gz
-
-2. Clone the Mesos git [repository](git-wip-us.apache.org/repos/asf/mesos.git) (***Advanced Users Only***)
-
-        $ git clone http://git-wip-us.apache.org/repos/asf/mesos.git
-
-## System Requirements
-
--  Mesos runs on Linux and Mac OSX.
-
--  Following are the instructions for stock Ubuntu 12.04 64 Bit. If you are using a different OS please install the packages accordingly.
-
-        # Ensure apt-get is up to date.
-        $ sudo apt-get update
-
-        # Install build tools.
-        $ sudo apt-get install build-essential
-
-        # Install OpenJDK java.
-        $ sudo apt-get install openjdk-6-jdk
-
-        # Install devel python.
-        $ sudo apt-get install python-dev
-
-        # Install devel libcurl (***Optional***).
-        $ sudo apt-get install libcurl4-nss-dev
-
-        # Install devel libsasl (***Only required for Mesos 0.14.0 or newer***).
-        $ sudo apt-get install libsasl2-dev
-
-
-If you are building from git repository, you will need to additionally install the following packages.
-
-        # Install autotoconf and automake.
-        $ sudo apt-get install autoconf
-
-        # Install libtool.
-        $ sudo apt-get install libtool
-
-***NOTES***
-
-> 1. The build process attempts to guess where your Java include directory is, but if you have set the `$JAVA_HOME` environment variable, it will use `$JAVA_HOME/include`, which may not be correct (or exist) on your machine (in which case you will see an error such as: `configure: error: failed to build against JDK (using libtool)`). If this is the case, we suggest you unset the `JAVA_HOME` environment variable.
-
-> 2. Mesos is currently being developed/tested/supported on 64 Bit machines only.
-
-
-## Building Mesos
-
-        # Change working directory.
-        $ cd mesos
-
-        # Bootstrap (***Skip this if you are not building from git repo***).
-        $ ./bootstrap
-
-        # Configure and build.
-        $ mkdir build
-        $ cd build
-        $ ../configure
-        $ make -j
-
-        # Run test suite.
-        $ make -j check
-
-        # Install (***Optional***).
-        $ make install
-
-## Examples
-Mesos comes bundled with example frameworks written in `C++`, `Java` and `Python`.
-
-        # Change into build directory.
-        $ cd build
-
-        # Start mesos master.
-        $ ./bin/mesos-master.sh --ip=127.0.0.1
-
-        # Start mesos slave.
-        $ ./bin/mesos-slave.sh --master=127.0.0.1:5050
-
-        # Visit the mesos web page.
-        $ http://127.0.0.1:5050
-
-        # Run C++ framework (***Exits after successfully running some tasks.***).
-        $ ./src/test-framework --master=127.0.0.1:5050
-
-        # Run Java framework (***Exits after successfully running some tasks.***).
-        $ ./src/examples/java/test-framework 127.0.0.1:5050
-
-        # Run Python framework (***Exits after successfully running some tasks.***).
-        $ ./src/examples/python/test-framework 127.0.0.1:5050
-
-*NOTE: To build the example frameworks, make sure you build the test suite by doing `make check`.*
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/High-Availability.md
----------------------------------------------------------------------
diff --git a/docs/High-Availability.md b/docs/High-Availability.md
deleted file mode 100644
index 77af5d5..0000000
--- a/docs/High-Availability.md
+++ /dev/null
@@ -1,50 +0,0 @@
----
-layout: documentation
----
-
-# High Availability
-
-Mesos can run in high-availability mode, in which multiple Mesos masters run simultaneously. In this mode there is only one active master, and the others masters act as stand-by replacements. These non-active masters will take over if the active master fails.
-
-Mesos uses Apache ZooKeeper in to coordinate the leader election of masters.
-
-Running mesos in this mode requires ZooKeeper. Mesos is automatically built with an included ZooKeeper client library.
-
-Leader election and detection in Mesos is done via ZooKeeper. See: http://zookeeper.apache.org/doc/trunk/recipes.html#sc_leaderElection for more information how ZooKeeper leader election works.
-
-## Use
-First, a ZooKeeper cluster must be running, and a znode should be created for exclusive use by Mesos:
-
-* Ensure the ZooKeeper cluster is up and running.
-* Create a znode for exclusive use by mesos. The znode path will need to be provided to all slaves / scheduler drivers.
-
-In order to spin up a Mesos cluster using multiple masters for fault-tolerance:
-
-* Start the mesos-master binaries using the `--zk` flag, e.g. `--zk=zk://host1:port1/path,host2:port2/path,...`
-* Start the mesos-slave binaries with `--master=zk://host1:port1/path,host2:port2/path,...`
-* Start any framework schedulers using the same zk path, the SchedulerDriver must be constructed with this path.
-
-Refer to the Scheduler API for how to deal with leadership changes.
-
-## Semantics
-The detector is implemented in the `src/detector` folder. In particular, we watch for several ZooKeeper session events:
-
-* Connection
-* Reconnection
-* Session Expiration
-* ZNode creation, deletion, updates
-
-We also explicitly timeout our sessions, when disconnected from ZooKeeper for an amount of time, see: `ZOOKEEPER_SESSION_TIMEOUT`. This is because the ZooKeeper client libraries only notify of session expiration upon reconnection. These timeouts are of particular interest in the case of network partitions.
-
-## Network Partitions
-When a network partition occurs, if a particular component is disconnected from ZooKeeper, the Master Detector of the partitioned component will induce a timeout event. This causes the component to be notified that there is no leading master.
-
-When slaves are master-less, they ignore incoming messages from masters to ensure that we don't act on a non-leading master's decision.
-
-When masters enter a leader-less state, they commit suicide.
-
-The following semantics are enforced:
-
-* If a slave is partitioned from the master, it will fail health-checks. The master will mark the slave as deactivated and send its tasks to LOST.
-* Deactivated slaves may not re-register with the master, and are instructed to shut down upon any further communication after deactivation.
-* When a slave is partitioned from ZooKeeper, it will enter a master-less state. It will ignore any master messages until a new master is elected.
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Home.md
----------------------------------------------------------------------
diff --git a/docs/Home.md b/docs/Home.md
deleted file mode 100644
index 0cbe85c..0000000
--- a/docs/Home.md
+++ /dev/null
@@ -1,44 +0,0 @@
----
-layout: documentation
----
-
-# Documentation
-
-## Users Documentation
-
-* [Deploy Scripts](Deploy-Scripts.md) for launching a Mesos cluster on a set of machines.
-* [EC2 scripts](EC2-Scripts.md) for launching a Mesos cluster on Amazon EC2.
-* [Logging and Debugging](Logging-and-Debugging.md) -- viewing Mesos and framework logs.
-* [High Availability](High-Availability) for running multiple masters simultaneously.
-* [Slave Recovery](Slave-Recovery.md)
-* [Using Linux Containers](Using-Linux-Containers.md) for resource isolation on slaves.
-* [Using Mesos Submit Tool](Using-the-mesos-submit-tool.md) for launching a framework with a single task.
-
-## Running Mesos Frameworks
-
-* [Running Hadoop on Mesos](https://github.com/mesos/hadoop)
-* [Running Spark on Mesos](https://cwiki.apache.org/confluence/display/SPARK/)
-* [Running Torque or MPI on Mesos](Running-torque-or-mpi-on-mesos.md)
-* [Running Jenkins on Mesos](https://github.com/jenkinsci/mesos-plugin)
-* [Running Chronos on Mesos](https://github.com/airbnb/chronos)
-* [Running Marathon on Mesos](https://github.com/mesosphere/marathon)
-* [Using Mesos with Hypertable on EC2](https://code.google.com/p/hypertable/wiki/Mesos)
-
-## Developing Mesos Frameworks and Applications
-
-* [Mesos Architecture](Mesos-Architecture.md) providing an overview of Mesos concepts.
-* [Mesos Code Internals](Mesos-Code-Internals.md) overview of the codebase and internal organization.
-* [Framework Development Guide](App-Framework-Development-Guide.md) describes how to build applications on top of Mesos.
-* [Mesos Command-Line Flags](Configuration.md) detailing various flags that can be set.
-
-## Contributing to Mesos
-
-* [Mesos Developers Guide](Mesos-developers-guide.md) includes resources for developers contributing to Mesos and the process of submitting patches for review.
-* [C++ Style Guide](Mesos-c++-style-guide.md)
-* [Mesos Development Road Map](Mesos-Roadmap.md)
-
-## More Info about Mesos
-
-* [Powered by Mesos](Powered-by-Mesos.md) lists organizations and software that are powered by Apache Mesos.
-* [Video and Slides of Mesos Presentations](Mesos-Presentations.md)
-* Academic Papers and Project History
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Logging-and-Debugging.md
----------------------------------------------------------------------
diff --git a/docs/Logging-and-Debugging.md b/docs/Logging-and-Debugging.md
deleted file mode 100644
index 9908dca..0000000
--- a/docs/Logging-and-Debugging.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-layout: page
----
-
-# Logging and Debugging
-
-Mesos uses the "[Google Logging library](http://code.google.com/p/google-glog)" and writes logs to @MESOS_HOME/logs@ by default, where @MESOS_HOME@ is the location where Mesos is installed. The log directory can be "configured":Configuration.textile using the @log_dir@ parameter.
-
-Frameworks that run on Mesos have their output stored to a "work" directory on each machine. By default, this is @MESOS_HOME/work@. Within this directory, a framework's output is placed in files called @stdout@ and @stderr@ in a directory of the form @slave-X/fw-Y/Z@, where X is the slave ID, Y is the framework ID, and multiple subdirectories Z are created for each attempt to run an executor for the framework. These files can also be accessed via the web UI of the slave daemon.
-
-Finally, using an "[alpha code branch](https://github.com/mesos/mesos/tree/andyk-event-history-nowebui)" which (at the time of this writing) is still in an alpha state, structured logs of cluster activity can be written to a tab-separated text file or a SQLite database using the "event history" system.
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Mesos-Architecture.md
----------------------------------------------------------------------
diff --git a/docs/Mesos-Architecture.md b/docs/Mesos-Architecture.md
deleted file mode 100644
index 7b0c411..0000000
--- a/docs/Mesos-Architecture.md
+++ /dev/null
@@ -1,32 +0,0 @@
----
-layout: documentation
----
-
-# Mesos Architecture
-
-![Mesos Architecture](/images/architecture3.jpg)
-
-The above figure shows the main components of Mesos.  Mesos consists of a *master* daemon that manages *slave* daemons running on each cluster node, and *mesos applications* (also called *frameworks*) that run *tasks* on these slaves.
-
-The master enables fine-grained sharing of resources (cpu, ram, ...) across applications by making them *resource offers*. Each resource offer contains a list of <slave ID, resource1: amount1, resource2, amount2, ...>.  The master decides *how many* resources to offer to each framework according to a given organizational policy, such as fair sharing, or strict priority. To support a diverse set of policies, the master employs a modular architecture that makes it easy to add new allocation modules via a plugin mechanism.
-
-A framework running on top of Mesos consists of two components: a *scheduler* that registers with the master to be offered resources, and an *executor* process that is launched on slave nodes to run the framework's tasks (see the [App/Framework development guide](App-Framework-development-guide.md) for more details about application schedulers and executors). While the master determines **how many** resources are offered to each framework, the frameworks' schedulers select **which** of the offered resources to use. When a frameworks accepts offered resources, it passes to Mesos a description of the tasks it wants to run on them. In turn, Mesos launches the tasks on the corresponding slaves.
-
-## Example of resource offer
-
-The figure below shows an example of how a framework gets scheduled to run a task.
-
-![Mesos Architecture](/images/architecture-example.jpg)
-
-Let's walk through the events in the figure.
-
-1. Slave 1 reports to the master that it has 4 CPUs and 4 GB of memory free. The master then invokes the allocation policy module, which tells it that framework 1 should be offered all available resources.
-1. The master sends a resource offer describing what is available on slave 1 to framework 1.
-1. The framework's scheduler replies to the master with information about two tasks to run on the slave, using <2 CPUs, 1 GB RAM> for the first task, and <1 CPUs, 2 GB RAM> for the second task.
-1. Finally, the master sends the tasks to the slave, which allocates appropriate resources to the framework's executor, which in turn launches the two tasks (depicted with dotted-line borders in the figure). Because 1 CPU and 1 GB of RAM are still unallocated, the allocation module may now offer them to framework 2.
-
-In addition, this resource offer process repeats when tasks finish and new resources become free.
-
-While the thin interface provided by Mesos allows it to scale and allows the frameworks to evolve independently, one question remains: how can the constraints of a framework be satisfied without Mesos knowing about these constraints? For example, how can a framework achieve data locality without Mesos knowing which nodes store the data required by the framework? Mesos answers these questions by simply giving frameworks the ability to **reject** offers. A framework will reject the offers that do not satisfy its constraints and accept the ones that do.  In particular, we have found that a simple policy called delay scheduling, in which frameworks wait for a limited time to acquire nodes storing the input data, yields nearly optimal data locality.
-
-You can also read much more about the Mesos architecture in this [technical paper](http://mesos.berkeley.edu/mesos_tech_report.pdf).
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Mesos-Code-Internals.md
----------------------------------------------------------------------
diff --git a/docs/Mesos-Code-Internals.md b/docs/Mesos-Code-Internals.md
deleted file mode 100644
index 1606bdc..0000000
--- a/docs/Mesos-Code-Internals.md
+++ /dev/null
@@ -1,41 +0,0 @@
----
-layout: page
----
-
-# Mesos Code Internals
-
-## Top-level directories in the Mesos distribution
-
-* 3rdparty - Contains necessary open source third party software that Mesos leverages for things such as logging, etc.
-* docs - Documentation that's packaged/shipped with each release.
-* ec2 - Scripts for launching a Mesos cluster on EC2. See the wiki page on "EC2-Scripts".
-* frameworks - Included Mesos Frameworks. See the READMEs in each one. See the "App/Framework development guide":App-Framework-development-guide.textile for a crash course in how Mesos Frameworks get resources from the Mesos master.
-* include - Contains headers that contain the interfaces that Mesos users need in order to interact with Mesos (e.g. the Mesos Framework API)
-* src - Contains the entire Mesos source tree. See below for more details about the directories inside of @src@.
-
-## Mesos source code
-
-The Mesos source code (found in @MESOS_HOME/src@) is organized into the following hierarchy:
-
-* common - Shared source files (such as utilities and data structures).
-* deploy
-* detector
-* examples
-* exec
-* files
-* launcher
-* linux
-* local
-* log
-* logging
-* master - Source files specific to the mesos-master daemon.
-* mesos
-* messages
-* python
-* sasl
-* scaling
-* sched - Source files specific to the mesos-slave daemon.
-* slave
-* tests
-* webui
-* zookeeper
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Mesos-Presentations.md
----------------------------------------------------------------------
diff --git a/docs/Mesos-Presentations.md b/docs/Mesos-Presentations.md
deleted file mode 100644
index 22ac8d1..0000000
--- a/docs/Mesos-Presentations.md
+++ /dev/null
@@ -1,46 +0,0 @@
----
-layout: page
----
-
-# Video and Slides of Mesos Presentations
-_(Listed in date descending order)_
-
-### Datacenter Management with Mesos ([Video](http://www.youtube.com/watch?v=YB1VW0LKzJ4))
-Benjamin Hindman
-Presented August 29th, 2013 at [AMP Camp](http://ampcamp.berkeley.edu/3/)
-
-### Building a Framework on Mesos: A Case Study with Jenkins ([Video](http://www.youtube.com/watch?v=TPXw_lMTJVk))
-Vinod Kone
-Presented July 25, 2013 at [SF Mesos Meetup](http://www.meetup.com/Distributed-data-processing-with-Mesos/events/128585772/)
-
-### Hadoop on Mesos ([Video](http://www.youtube.com/watch?v=SFj5EMw8THk))
-Brenden Matthews
-Presented July 25, 2013 at [SF Mesos Meetup](http://www.meetup.com/Distributed-data-processing-with-Mesos/events/128585772/)
-
-### Introduction to Apache Mesos ([Slides](https://speakerdeck.com/benh/apache-mesos-nyc-meetup))
-Benjamin Hindman
-Presented August 20, 2013 at [NYC Mesos Meetup](https://mesos-nyc-aug2013.eventbrite.com/)
-
-### Chronos: A distributed, fault-tolerant and highly available job orchestration framework for Mesos ([Slides](https://speakerdeck.com/mesos/chronos-august-2013-nyc-meetup))
-Florian Leibert
-Presented August 20, 2013 at [NYC Mesos Meetup](https://mesos-nyc-aug2013.eventbrite.com/)
-
-### Airbnb tech talk ([Video](http://www.youtube.com/watch?v=Hal00g8o1iY))
-Benjamin Hindman
-Presented September 6, 2012 at [Airbnb](http://airbnb.com)
-
-### Managing Twitter clusters with Mesos ([Video](http://www.youtube.com/watch?v=37OMbAjnJn0))
-Benjamin Hindman
-Presented August 22, 2012 at [AMP Camp](http://ampcamp.berkeley.edu)
-
-### Mesos: A Platform for Fine-Grained Resource Sharing in Datacenters ([Video](http://www.youtube.com/watch?v=dB8IDu7g9Nc))
-Matei Zaharia
-Presented March 2011 at [UC Berkeley](http://berkeley.edu)
-
-### Mesos: Efficiently Sharing the Datacenter ([Video](http://vimeo.com/17821090))
-Benjamin Hindman
-Presented November 8, 2010 at [LinkedIn](http://linkedin.com)
-
-### Mesos: A Resource Management Platform for Hadoop and Big Data Clusters ([Video](http://www.youtube.com/watch?v=lE3jR6nM3bw))
-Matei Zaharia
-Presented Summer 2010 at [Yahoo](http://yahoo.com)
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Mesos-Roadmap.md
----------------------------------------------------------------------
diff --git a/docs/Mesos-Roadmap.md b/docs/Mesos-Roadmap.md
deleted file mode 100644
index a2e2750..0000000
--- a/docs/Mesos-Roadmap.md
+++ /dev/null
@@ -1,7 +0,0 @@
----
-layout: page
----
-
-# Mesos Roadmap
-
-The [Mesos JIRA Road Map](https://issues.apache.org/jira/browse/MESOS#selectedTab=com.atlassian.jira.plugin.system.project%3Aroadmap-panel) provides the most up-to-date information regarding what is planned in future releases. If you have comments or suggestions, you may direct them to the Mesos dev list (dev@mesos.apache.org), or assign issues to future releases.
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Mesos-c++-style-guide.md
----------------------------------------------------------------------
diff --git a/docs/Mesos-c++-style-guide.md b/docs/Mesos-c++-style-guide.md
deleted file mode 100644
index e293ee1..0000000
--- a/docs/Mesos-c++-style-guide.md
+++ /dev/null
@@ -1,72 +0,0 @@
----
-layout: page
----
-
-# Mesos C++ Style Guide
-
-The Mesos codebase follows the [Google C++ Style Guide](http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml) with the following differences:
-
-## Naming
-
-### Variable Names
-* We use [lowerCamelCase](http://en.wikipedia.org/wiki/CamelCase#Variations_and_synonyms) for variable names (Google uses snake_case, and their class member variables have trailing underscores).
-
-### Constant Names
-* We use lowerCamelCase for constant names (Google uses a `k` followed by mixed case, e.g. `kDaysInAWeek`).
-
-### Function Names
-* We use lowerCamelCase for function names (Google uses mixed case for regular functions; and their accessors and mutators match the name of the variable).
-
-## Strings
-* Strings used in log and error messages should end without a period.
-
-## Comments
-* End each sentence with a period.
-* At most 70 characters per line in comments.
-
-## Indentation
-* Newline when calling or defining a function: indent with 4 spaces.
-* We do not follow Google's style of wrapping on the open parenthesis, the general goal is to reduce visual "jaggedness" in the code. Prefer (1), (4), (5), sometimes (3), never (2):
-
-<pre>
-// 1: OK.
-allocator->resourcesUnused(frameworkId, slaveId, resources, filters);
-
-// 2: Don't use.
-allocator->resourcesUnused(frameworkId, slaveId,
-                           resources, filters);
-
-// 3: Don't use in this case due to "jaggedness".
-allocator->resourcesUnused(frameworkId,
-                           slaveId,
-                           resources,
-                           filters);
-
-// 3: In this case, 3 is OK.
-foobar(someArgument,
-       someOtherArgument,
-       theLastArgument);
-
-// 4: OK.
-allocator->resourcesUnused(
-    frameworkId,
-    slaveId,
-    resources,
-    filters);
-
-// 5: OK.
-allocator->resourcesUnused(
-    frameworkId, slaveId, resources, filters);
-</pre>
-
-* Newline for an assignment statement: indent with 2 spaces.
-
-<pre>
-Try&lt;Duration&gt; failoverTimeout =
-  Duration::create(FrameworkInfo().failover_timeout());
-</pre>
-
-## New Lines
-* 1 blank line at the end of the file.
-* Elements outside classes (classes, structs, global functions, etc.) should be spaced apart by 2 blank lines.
-* Elements inside classes (member variables and functions) should not be spaced apart by more than 1 blank line.
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Mesos-developers-guide.md
----------------------------------------------------------------------
diff --git a/docs/Mesos-developers-guide.md b/docs/Mesos-developers-guide.md
deleted file mode 100644
index a0276c7..0000000
--- a/docs/Mesos-developers-guide.md
+++ /dev/null
@@ -1,72 +0,0 @@
----
-layout: page
----
-
-# Developers Guide
-
-## Reporting a Bug
-* You've found a bug and want to report it. THANKS!
-* We track all issues via Apache's hosted [JIRA issue tracker](https://issues.apache.org/jira/browse/MESOS), so if you don't already have an account you'll need to register for one. It's quick and easy.
-* A JIRA should be created for every task, feature, bug-fix, etc. This makes it easy to track track progress of tickets.
-* Assign the JIRA issue to yourself before you start working on it. This helps to avoid duplication of work, and alerts anyone who is following that issue.
-
-## Submitting a patch
-* You've fixed a bug or added a feature and want to contribute it. AWESOME!
-* We use [Apache Review Board](https://reviews.apache.org) for code reviews. If you don't already have an account, you'll need to create one (it's separate from your Apache JIRA account).
-* A code review request should be created for every JIRA that involves a change to the codebase.
-
-### Before you submit
-1. Check out the code from the Apache repository via Git or SVN. Instructions are on the [Getting Started](http://mesos.apache.org/gettingstarted/) page.
-
-1. Join the dev@mesos.apache.org mailing list by sending an email to dev-subscribe@mesos.apache.org
-
-1. Find a JIRA that is currently unassigned that you want to work on at http://issues.apache.org/jira/browse/MESOS, or create your own (you'll need a JIRA account for this, see below)!
-    2. This could be a JIRA representing a bug (possibly a bug that you encountered and reported, e.g. when trying to build) or a new feature
-
-1. Assign the JIRA to yourself. To do this, you will need:
-    2. An Apache JIRA user account (sign up for one [here](https://issues.apache.org/jira/secure/Signup!default.jspa))
-    2. You need to be added to the list of Mesos "contributors" by a Mesos committer (send email to dev@mesos.apache.org) in order to be assigned (or to assign yourself) to a JIRA issue
-
-1. Formulate a plan for resolving the issue, propose your plan via comments in the JIRA
-
-1. Create one or more test cases to exercise the bug or the feature (the Mesos team uses [test-driven development](http://en.wikipedia.org/wiki/Test-driven_development)), before you start coding, make sure these test cases all fail.
-
-1. Make your changes to the code (using whatever IDE/editor you choose) to actually fix the bug or implement the feature.
-    2. Before beginning, please read the [Mesos C++ Style Guide](Mesos-c++-style-guide.md)
-    2. Most of your changes will probably be to files inside of &lt;BASE_MESOS_DIR&gt;/src
-    2. To build, we recommend that you don't build inside of the src directory. We recommend you do the following:
-        3. From inside of the root Mesos directory: `mkdir build && cd build`
-        3. `../configure`
-        3. `make`
-        3. Now all of the files generated by the build process will be contained in the build directory you created, instead of being spread throughout the src directory, which is a bit messier. This is both cleaner, and makes it easy to clean up if you want to get rid of the files generated by `configure` and `make`. I.e. You can reset your build process without risking changes you made in the src directory, by simply deleting the build directory, and creating a new one.
-
-1. Make sure all of your test cases now pass.
-	2. `make check`
-
-1. Make sure to pull in any changes that have been committed to master branch. Using Git, do this via something like:
-  2. `git checkout master`
-  2. `git pull`
-  3. `git co my_branch`
-  2. Check the output of `git diff master` and make sure it lists only your changes. If other changes you did not make are listed, try a rebase to bring your branch up to date with master.
-
-### Submit your patch
-1. You're ready to submit your patch for review!
-	2. Log in or create an account at ]Apache Review Board](http://reviews.apache.org)
-    2. The easiest (and recommended) way to submit reviews is through `post-reviews.py` a wrapper around post-review.
-	2. First, install post-review. [See Instructions](http://www.reviewboard.org/docs/manual/dev/users/tools/post-review/)
-	2. From your local branch run `support/post-reviews.py`.
-	2. Note that `post-reviews.py` creates a new review for every commit on your branch that is different from the `master`.
-    2. Be sure to add your JIRA issue id (e.g. MESOS-01) to the field labeled "Bugs" (this will automatically link)
-    2. Under "Description" in addition to details about your changes, include a description of any wiki documentation pages need to be added, or are affected by your changes (e.g. did you change or add any configuration options/flags? Did you add a new binary?)
-
-1. Wait for a code review from another Mesos developer via Review Board, address their feedback and upload updated patches until you receive a "Ship It" from a Mesos committer.
-    2. Review Board comments should be used for code-specific discussions, and JIRA comments for bigger-picture design discussions.
-    2. Always respond to each RB comment that you address directly (i.e. each comment can be responded to directly) with either "Done." or a comment explaining how you addressed it.
-    2. If an issue has been raised in the review, please resolve the issue as "Fixed" or "Dropped". If "Dropped" please add a comment explaining the reason. Also, if your fix warrants a comment (e.g., fixed differently than suggested by the reviewer) please add a comment.
-
-1. After consensus is reached on your JIRA/patch, you're review request will receive a "Ship It!" from a committer, and then a committer will commit your patch to the git repository. Congratulations and thanks for participating in our community!
-
-1. The last step is to ensure that the necessary documentation gets created or updated so the whole world knows about your new feature or bug fix.
-
-## Style Guides
-* For patches to the core, we ask that you follow the [Mesos C++ Style Guide](Mesos-c++-style-guide.md)
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Powered-by-Mesos.md
----------------------------------------------------------------------
diff --git a/docs/Powered-by-Mesos.md b/docs/Powered-by-Mesos.md
deleted file mode 100644
index b91bd22..0000000
--- a/docs/Powered-by-Mesos.md
+++ /dev/null
@@ -1,29 +0,0 @@
----
-layout: page
----
-
-# Powered By Mesos
-
-## Organizations using Mesos
-
-* [Airbnb](http://www.airbnb.com)
-* [Categorize](http://categorize.co)
-* [CloudPhysics](http://cloudphysics.com)
-* [Conviva](http://www.conviva.com)
-* [iQIYI](http://www.iqiyi.com/)
-* [MediaCrossing](http://www.mediacrossing.com)
-* [Sharethrough](http://www.sharethrough.com)
-* [Twitter](http://www.twitter.com)
-* [UCSF](http://www.ucsf.edu)
-* [UC Berkeley](http://www.berkeley.edu)
-* [Xogito](http://www.xogito.com)
-
-## Software projects built on Mesos
-
-* [Spark](http://spark.incubator.apache.org/) cluster computing framework
-Spark
-http://spark.incubator.apache.org/index.html
-* [Chronos](https://github.com/airbnb/chronos)
-* [Marathon](https://github.com/mesosphere/marathon)
-
-If you're using Mesos, please add yourself to the list above, or email user@mesos.apache.org and we'll add you!
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Running-a-web-application-farm-on-mesos.md
----------------------------------------------------------------------
diff --git a/docs/Running-a-web-application-farm-on-mesos.md b/docs/Running-a-web-application-farm-on-mesos.md
deleted file mode 100644
index c81ae77..0000000
--- a/docs/Running-a-web-application-farm-on-mesos.md
+++ /dev/null
@@ -1,25 +0,0 @@
----
-layout: page
----
-
-# Running a web-farm on Mesos in your private cluster
-
-Mesos comes with a framework written in Python that will run a web load balancer ([HAProxy](http://haproxy.1wt.eu)) and use apache for the backend servers. The framework is in MESOS_HOME/frameworks/haproxy+apache. There is a README in that directory that might be slightly helpful, but we recommend looking a the framework scheduler and executor code to familiarize yourself with the concepts behind the framework.
-
-HAProxy has this annoying artifact that in order to ramp up and down in the number of servers it is load balancing between it has to kill the existing running HAProxy process and start a new one, which inevitably leads to dropped connections. Some people have looked into ways to get around the dropped connections (see [Animoto blog post about adding/removing servers to haproxy](http://labs.animoto.com/2010/03/10/uptime-with-haproxy)).
-
-We recommend checking out other load balancers as well. Basically, whatever load balancer you're using, the framework scheduler should query the statistics of the load balancer to figure out how many web requests are being served to it and then ramp up and down the number of web server tasks based on those statistics. We've also written a framework that uses [Linux Virtual Server (LVS)](http://www.linux-vs.org) as the load balancer. LVS can be run in three different modes, and any should work with Mesos.
-
-## Running a web farm on Mesos on EC2
-
-### Why use Mesos on EC2?
-
-Many might wonder what the advantages of running a Mesos web farm on EC2 are over simply using the Amazon Elastic Load Balancer which will automatically ramp up and down the number of back end web server AMIs for you. We have found that by running Mesos on top of EC2 we are able to share a single EC2 instance between our web frameworks and our other frameworks such as Hadoop or Spark which can be used to do data analytics on top of the click logs being generated by the web frameworks, for instance.
-
-We have also found that we can allow multiple different users share an EC2 allocation of instances much more efficiently (and thus reduce EC2 costs) by using Mesos on that EC2 allocation.
-
-### Where to look to get started
-
-We have also successfully used Amazon's Elastic Load Balancer instead of HAProxy. This is still in a non-master development branch called [andyk-elb+apache-notorque](https://github.com/mesos/mesos/tree/andyk-elb+apache-notorque). You will need to set up a Mesos EC2 cluster and the framework uses python boto to connect to Amazon's CloudWatch and EC2 so you will need to create a credentials file for boto to be able to connect to those amazon services correctly from the node on which you run the framework.
-
-Watch out when using ELB however, since it displays some interesting and non-intuitive behavior when ramping up and down (though it doesn't drop connections like HAProxy does out of the box). [This blog post](http://shlomoswidler.com/2009/07/elastic-in-elastic-load-balancing-elb.html) details how ELB ramps up and down. 
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Running-torque-or-mpi-on-mesos.md
----------------------------------------------------------------------
diff --git a/docs/Running-torque-or-mpi-on-mesos.md b/docs/Running-torque-or-mpi-on-mesos.md
deleted file mode 100644
index 61fb24f..0000000
--- a/docs/Running-torque-or-mpi-on-mesos.md
+++ /dev/null
@@ -1,9 +0,0 @@
----
-layout: page
----
-
-# Torque or MPI on Mesos
-
-Torque is cluster scheduler that we have ported in a very simple way (via some Python wrappers around Torque) to run on top of Mesos. This port can be found in @MESOS_HOME/frameworks/torque@. There is a README.txt in that directory with some details. But for now while were in Alpha looking directly at the python source for the framework scheduler and executor will be helpful.
-
-We have also run MPICH2 directly on top of mesos. See @MESOS_HOME/frameworks/mpi@ for this port. Basically it sets up the MPICH2 MPD ring for you when you use nmpiexec (which is named that way because Mesos used to be called **N**exus).
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Slave-Recovery.md
----------------------------------------------------------------------
diff --git a/docs/Slave-Recovery.md b/docs/Slave-Recovery.md
deleted file mode 100644
index 928cfea..0000000
--- a/docs/Slave-Recovery.md
+++ /dev/null
@@ -1,66 +0,0 @@
----
-layout: page
----
-
-# Slave Recovery
-
-Slave recovery is a feature of Mesos that allows:
-
- 1. Executors/tasks to keep running when the slave process is down and
- 2. Allows a restarted slave process to reconnect with running executors/tasks on the slave.
-
-Mesos slave could be restarted for an upgrade or due to a crash. This feature is introduced in ***0.14.0*** release.
-
-## How does it work?
-
-Slave recovery works by having the slave checkpoint enough information (e.g., Task Info, Executor Info, Status Updates) about the running tasks and executors to local disk. Once the slave ***and*** the framework(s) enable checkpointing, any subsequent slave restarts would recover the checkpointed information and reconnect with the executors. Note that if the host running the slave process is rebooted all the executors/tasks are killed.
-
-> NOTE: To enable slave recovery both the slave and the framework should explicitly request checkpointing.
-> Alternatively, a framework that doesn't want the disk i/o overhead of checkpointing can opt out of checkpointing.
-
-
-## Enabling slave checkpointing
-
-As part of this feature, 4 new flags were added to the slave.
-
-* `checkpoint` :  Whether to checkpoint slave and frameworks information
-                  to disk [Default: false].
-    - This enables a restarted slave to recover status updates and reconnect
-      with (--recover=reconnect) or kill (--recover=kill) old executors.
-
-	* `strict` : Whether to do recovery in strict mode [Default: true].
-		* If strict=true, any and all recovery errors are considered fatal.
-		* If strict=false, any errors (e.g., corruption in checkpointed data) during recovery are
-      ignored and as much state as possible is recovered.
-
-	* `recover` : Whether to recover status updates and reconnect with old executors [Default: reconnect].
-		* If recover=reconnect, Reconnect with any old live executors.
-		* If recover=cleanup, Kill any old live executors and exit.
-      Use this option when doing an incompatible slave or executor upgrade!).
-      NOTE: If no checkpointing information exists, no recovery is performed
-      and the slave registers with the master as a new slave.
-
-	* `recovery_timeout` : Amount of time allotted for the slave to recover [Default: 15 mins].
-		* If the slave takes longer than `recovery_timeout` to recover, any executors that are waiting to
-    reconnect to the slave will self-terminate.
-    NOTE: This flag is only applicable when `--checkpoint` is enabled.
-
-> NOTE: If checkpointing is enabled on the slave, but none of the frameworks have enabled checkpointing,
-> executors/tasks of frameworks die when the slave dies and are not recovered.
-
-A restarted slave should re-register with master within a timeout (currently, 75s). If the slave takes longer
-than this timeout to re-register, the master shuts down the slave, which in turn shuts down any live executors/tasks.
-Therefore, it is highly recommended to automate the process of restarting a slave (e.g, using [monit](http://mmonit.com/monit/)).
-
-**For the complete list of slave options: ./mesos-slave.sh --help**
-
-## Enabling framework checkpointing
-
-As part of this feature, `FrameworkInfo` has been updated to include an optional `checkpoint` field. A framework that would like to opt in to checkpointing should set `FrameworkInfo.checkpoint=True` before registering with the master.
-
-> NOTE: Frameworks that have enabled checkpointing will only get offers from checkpointing slaves. So, before setting `checkpoint=True` on FrameworkInfo, ensure that there are slaves in your cluster that have enabled checkpointing.
-> Because, if there are no checkpointing slaves, the framework would not get any offers and hence cannot launch any tasks/executors!
-
-## Upgrading to 0.14.0
-
-If you want to upgrade a running Mesos cluster to 0.14.0 to take advantage of slave recovery please follow the [upgrade instructions](https://github.com/apache/mesos/blob/master/docs/Upgrades.md).
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Upgrades.md
----------------------------------------------------------------------
diff --git a/docs/Upgrades.md b/docs/Upgrades.md
deleted file mode 100644
index eb74b24..0000000
--- a/docs/Upgrades.md
+++ /dev/null
@@ -1,58 +0,0 @@
----
-layout: page
----
-
-# Upgrading Mesos
-This document serves as a guide for users who wish to upgrade an existing mesos cluster. Some versions require particular upgrade techniques when upgrading a running cluster. Some upgrades will have incompatible changes.
-
-## Upgrading from 0.14.0 to 0.15.0.
-
-In order to upgrade a running cluster:
-
-* Install the new master binaries.
-* Restart the masters with --credentials pointing to credentials of the framework(s).
-* NOTE: --authentication=false (default) allows both authenticated and unauthenticated frameworks to register.
-* Install the new slave binaries and restart the slaves.
-* Upgrade the executors by linking the latest native library and mesos jar (if necessary).
-* Upgrade the schedulers by linking the latest native library and mesos jar (if necessary).
-* NOTE: Schedulers should implement the new `reconcileTasks` driver method.
-* Schedulers should call the new `MesosSchedulerDriver` constructor that takes `Credential` to authenticate.
-* Restart the schedulers.
-* Restart the masters with --authentication=true.
-* NOTE: After the restart unauthenticated frameworks *will not* be allowed to register.
-
-
-## Upgrading from 0.13.0 to 0.14.0.
-
-In order to upgrade a running cluster:
-
-* Install the new master binaries and restart the masters.
-* NOTE: /vars endpoint has been removed.
-* Upgrade the executors by linking the latest native library and mesos jar (if necessary).
-* Install the new slave binaries.
-* Restart the slaves after adding --checkpoint flag to enable checkpointing.
-* NOTE: /vars endpoint has been removed.
-* Upgrade the schedulers by linking the latest native library and mesos jar (if necessary).
-* Set FrameworkInfo.checkpoint in the scheduler if checkpointing is desired (recommended).
-* Restart the schedulers.
-* Restart the masters (to get rid of the cached FrameworkInfo).
-* Restart the slaves (to get rid of the cached FrameworkInfo).
-
-## Upgrading from 0.12.0 to 0.13.0.
-In order to upgrade a running cluster:
-
-* Install the new master binaries and restart the masters.
-* Upgrade the schedulers by linking the latest native library and mesos jar (if necessary).
-* Restart the schedulers.
-* Install the new slave binaries.
-* NOTE: cgroups_hierarchy_root slave flag is renamed as cgroups_hierarchy
-* Restart the slaves.
-* Upgrade the executors by linking the latest native library and mesos jar (if necessary).
-
-## Upgrading from 0.11.0 to 0.12.0.
-In order to upgrade a running cluster:
-
-* Install the new slave binaries and restart the slaves.
-* Install the new master binaries and restart the masters.
-
-If you are a framework developer, you will want to examine the new 'source' field in the ExecutorInfo protobuf. This will allow you to take further advantage of the resource monitoring.

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/Using-the-mesos-submit-tool.md
----------------------------------------------------------------------
diff --git a/docs/Using-the-mesos-submit-tool.md b/docs/Using-the-mesos-submit-tool.md
deleted file mode 100644
index cc115e2..0000000
--- a/docs/Using-the-mesos-submit-tool.md
+++ /dev/null
@@ -1,9 +0,0 @@
----
-layout: page
----
-
-# Using the Mesos Submit Tool
-
-Sometimes you just want to run a command or launch a binary on all (or a subset) of the nodes in your cluster. `mesos-submit` lets you do just that!
-
-Mesos-submit is a little framework that lets you run a binary in the Mesos cluster without having to keep a scheduler running on the machine you submitted it from. You call mesos-submit <binary> and the script will launch a framework with a single task. This task then takes over as the scheduler for this framework (using the scheduler failover feature), and the mesos-submit process (which was the initial scheduler) can safely exit. The task then goes on to run the command. This is useful for people who want to submit their schedulers to the cluster for example.
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/mesos/blob/36a78668/docs/allocation-module.md
----------------------------------------------------------------------
diff --git a/docs/allocation-module.md b/docs/allocation-module.md
new file mode 100644
index 0000000..b308a23
--- /dev/null
+++ b/docs/allocation-module.md
@@ -0,0 +1,132 @@
+---
+layout: documentation
+---
+
+# Mesos Allocation Module
+
+The logic that the Mesos master uses to determine which frameworks to make offer resource offers to is encapsulated in the Master's _allocation module_.  The allocation module is a pluggable component that organizations can use to implement their own sharing policy, e.g. fair-sharing, Dominant Resource Fairness (see [the DRF paper](http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-55.pdf)), priority, etc.
+
+## Allocation Module API
+
+Mesos is implemented in C++, so allocation modules are implemented in C++, and inherit the @AllocatorProcess@ class defined in @MESOS_HOME/src/master/allocator.hpp@. As of the time of this writing (5/29/13), the API for allocation modules is as follows:
+
+```
+  virtual ~AllocatorProcess() {}
+
+  virtual void initialize(
+      const Flags& flags,
+      const process::PID<Master>& master,
+      const hashmap<std::string, RoleInfo>& roles) = 0;
+
+  virtual void frameworkAdded(
+      const FrameworkID& frameworkId,
+      const FrameworkInfo& frameworkInfo,
+      const Resources& used) = 0;
+
+  virtual void frameworkRemoved(
+      const FrameworkID& frameworkId) = 0;
+
+  virtual void frameworkActivated(
+      const FrameworkID& frameworkId,
+      const FrameworkInfo& frameworkInfo) = 0;
+
+  virtual void frameworkDeactivated(
+      const FrameworkID& frameworkId) = 0;
+
+  virtual void slaveAdded(
+      const SlaveID& slaveId,
+      const SlaveInfo& slaveInfo,
+      const hashmap<FrameworkID, Resources>& used) = 0;
+
+  virtual void slaveRemoved(
+      const SlaveID& slaveId) = 0;
+
+  virtual void updateWhitelist(
+      const Option<hashset<std::string> >& whitelist) = 0;
+
+  virtual void resourcesRequested(
+      const FrameworkID& frameworkId,
+      const std::vector<Request>& requests) = 0;
+
+  // Whenever resources offered to a framework go unused (e.g.,
+  // refused) the master invokes this callback.
+  virtual void resourcesUnused(
+      const FrameworkID& frameworkId,
+      const SlaveID& slaveId,
+      const Resources& resources,
+      const Option<Filters>& filters) = 0;
+
+  // Whenever resources are "recovered" in the cluster (e.g., a task
+  // finishes, an offer is removed because a framework has failed or
+  // is failing over) the master invokes this callback.
+  virtual void resourcesRecovered(
+      const FrameworkID& frameworkId,
+      const SlaveID& slaveId,
+      const Resources& resources) = 0;
+
+  // Whenever a framework that has filtered resources wants to revive
+  // offers for those resources the master invokes this callback.
+  virtual void offersRevived(
+      const FrameworkID& frameworkId) = 0;
+```
+
+The default allocation module is the HierarchicalAllocatorProcess, which can be found in @MESOS_HOME/src/master/hierarchical_allocator_process.hpp@. You can reference this as a starting place if you choose to write your own allocation module.
+
+## Sorter API
+
+Additionally, the hierarchical allocator module can be extended without the need to reimplement the entirety of the allocation logic through the use of the @Sorter@ abstraction.
+
+Sorters define the order that roles or frameworks should be offered resources in by taking "client" objects and some information about those clients and returning an ordered list of clients.
+
+Sorters are implemented in C++ and inherit the @Sorter@ class defined in @MESOS_HOME/src/master/sorter.hpp@. As of the time of this writing, the API for Sorters is as follows:
+
+```
+  virtual ~Sorter() {}
+
+  // Adds a client to allocate resources to. A client
+  // may be a user or a framework.
+  virtual void add(const std::string& client, double weight = 1) = 0;
+
+  // Removes a client.
+  virtual void remove(const std::string& client) = 0;
+
+  // Readds a client to the sort after deactivate.
+  virtual void activate(const std::string& client) = 0;
+
+  // Removes a client from the sort, so it won't get allocated to.
+  virtual void deactivate(const std::string& client) = 0;
+
+  // Specify that resources have been allocated to the given client.
+  virtual void allocated(const std::string& client,
+                         const Resources& resources) = 0;
+
+  // Specify that resources have been unallocated from the given client.
+  virtual void unallocated(const std::string& client,
+                           const Resources& resources) = 0;
+
+  // Returns the resources that have been allocated to this client.
+  virtual Resources allocation(const std::string& client) = 0;
+
+  // Add resources to the total pool of resources this
+  // Sorter should consider.
+  virtual void add(const Resources& resources) = 0;
+
+  // Remove resources from the total pool.
+  virtual void remove(const Resources& resources) = 0;
+
+  // Returns a list of all clients, in the order that they
+  // should be allocated to, according to this Sorter's policy.
+  virtual std::list<std::string> sort() = 0;
+
+  // Returns true if this Sorter contains the specified client,
+  // either active or deactivated.
+  virtual bool contains(const std::string& client) = 0;
+
+  // Returns the number of clients this Sorter contains,
+  // either active or deactivated.
+  virtual int count() = 0;
+```
+
+The default @Sorter@ is the DRFSorter, which implements fair sharing and can be found at @MESOS_HOME/src/master/drf_sorter.hpp@.
+
+For DRF, if weights are specified in Sorter::add, a client's share will be divided by the weight, creating a form of priority. For example, a role that has a weight of 2 will be offered twice as many resources as a role with weight 1.
\ No newline at end of file


Mime
View raw message