mesos-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dles...@apache.org
Subject svn commit: r1661841 [3/3] - in /mesos/site: publish/ publish/documentation/app-framework-development-guide/ publish/documentation/authorization/ publish/documentation/configuration/ publish/documentation/getting-started/ publish/documentation/latest/a...
Date Tue, 24 Feb 2015 05:22:08 GMT
Modified: mesos/site/source/documentation/latest/app-framework-development-guide.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/app-framework-development-guide.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/app-framework-development-guide.md (original)
+++ mesos/site/source/documentation/latest/app-framework-development-guide.md Tue Feb 24 05:22:07
2015
@@ -79,12 +79,15 @@ Declared in `MESOS_HOME/include/mesos/sc
   /**
    * Invoked when the status of a task has changed (e.g., a slave is
    * lost and so the task is lost, a task finishes and an executor
-   * sends a status update saying so, etc). Note that returning from
-   * this callback _acknowledges_ receipt of this status update! If
-   * for whatever reason the scheduler aborts during this callback (or
+   * sends a status update saying so, etc). If implicit
+   * acknowledgements are being used, then returning from this
+   * callback _acknowledges_ receipt of this status update! If for
+   * whatever reason the scheduler aborts during this callback (or
    * the process exits) another status update will be delivered (note,
    * however, that this is currently not true if the slave sending the
-   * status update is lost/fails during that time).
+   * status update is lost/fails during that time). If explicit
+   * acknowledgements are in use, the scheduler must acknowledge this
+   * status on the driver.
    */
   virtual void statusUpdate(SchedulerDriver* driver,
                             const TaskStatus& status) = 0;

Modified: mesos/site/source/documentation/latest/authorization.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/authorization.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/authorization.md (original)
+++ mesos/site/source/documentation/latest/authorization.md Tue Feb 24 05:22:07 2015
@@ -140,7 +140,7 @@ There are couple of important things to
 7. Framework `foo` can only register with `analytics` role but no other roles. Also, no other
framework can register with any roles.
 
             {
-              "permissive" : "false",
+              "permissive" : false,
 
               "register_frameworks": [
                                        {
@@ -154,7 +154,7 @@ There are couple of important things to
 8. Only `ops` principal can shutdown any frameworks through "/shutdown" HTTP endpoint.
 
             {
-              "permissive" : "false",
+              "permissive" : false,
 
               "shutdown_frameworks": [
                                        {

Modified: mesos/site/source/documentation/latest/configuration.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/configuration.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/configuration.md (original)
+++ mesos/site/source/documentation/latest/configuration.md Tue Feb 24 05:22:07 2015
@@ -7,7 +7,7 @@ layout: documentation
 
 The Mesos master and slave can take a variety of configuration options through command-line
arguments, or environment variables. A list of the available options can be seen by running
`mesos-master --help` or `mesos-slave --help`. Each option can be set in two ways:
 
-* By passing it to the binary using `--option_name=value`.
+* By passing it to the binary using `--option_name=value`, either specifying the value directly,
or specifying a file in which the value resides (`--option_name=file://path/to/file`). The
path can be absolute or relative to the current working directory.
 * By setting the environment variable `MESOS_OPTION_NAME` (the option name with a `MESOS_`
prefix added to it).
 
 Configuration values are searched for first in the environment, then on the command-line.
@@ -158,8 +158,7 @@ If you have special compilation requirem
       ZooKeeper URL (used for leader election amongst masters)
       May be one of:
 <pre><code>zk://host1:port1,host2:port2,.../path
-zk://username:password@host1:port1,host2:port2,.../path
-file://path/to/file (where file contains one of the above)</code></pre>
+zk://username:password@host1:port1,host2:port2,.../path</code></pre>
     </td>
   </tr>
 </table>
@@ -181,10 +180,9 @@ file://path/to/file (where file contains
       --acls=VALUE
     </td>
     <td>
-      The value could be a JSON formatted string of ACLs
-      or a file path containing the JSON formatted ACLs used
-      for authorization. Path could be of the form <code>file:///path/to/file</code>
-      or <code>/path/to/file</code>.
+      The value is a JSON formatted string of ACLs. Remember you can also use
+      the <code>file:///path/to/file</code> argument value format to write the
+      JSON in a file.
       <p/>
       See the ACLs protobuf in mesos.proto for the expected format.
       <p/>
@@ -267,7 +265,7 @@ file://path/to/file (where file contains
       Either a path to a text file with a list of credentials,
       each line containing 'principal' and 'secret' separated by whitespace,
       or, a path to a JSON-formatted file containing credentials.
-      Path could be of the form <code>file:///path/to/file</code> or <code>/path/to/file</code>.
+      Path should be of the form <code>file:///path/to/file</code>
       <p/>
       JSON file Example:
 <pre><code>{
@@ -324,8 +322,9 @@ file://path/to/file (where file contains
       subsystems.
       <p/>
       Use <code>--modules=filepath</code> to specify the list of modules via
a
-      file containing a JSON formatted string. 'filepath' can be
-      of the form <code>file:///path/to/file</code> or <code>/path/to/file</code>.
+      file containing a JSON formatted string. Remember you can also use
+      the <code>file:///path/to/file</code> argument value format to write the
+      JSON in a file.
       <p/>
       Use <code>--modules="{...}"</code> to specify the list of modules inline.
       <p/>
@@ -382,8 +381,9 @@ file://path/to/file (where file contains
       or a file path containing the JSON formatted rate limits used
       for framework rate limiting.
       <p/>
-      Path could be of the form <code>file:///path/to/file</code>
-        or <code>/path/to/file</code>.
+      Remember you can also use
+      the <code>file:///path/to/file</code> argument value format to write the
+      JSON in a file.
       <p/>
 
       See the RateLimits protobuf in mesos.proto for the expected format.
@@ -534,10 +534,15 @@ file://path/to/file (where file contains
       --whitelist=VALUE
     </td>
     <td>
-      Path to a file with a list of slaves
-      (one per line) to advertise offers for.
+      A filename which contains a list of slaves (one per line) to advertise
+      offers for. The file is watched, and periodically re-read to refresh
+      the slave whitelist. By default there is no whitelist / all machines are
+      accepted. (default: None)
+     <p/>
+
+      Example:
+      <pre><code>file:///etc/mesos/slave_whitelist</code></pre>
       <p/>
-      Path could be of the form <code>file:///path/to/file</code> or <code>/path/to/file</code>.
(default: *)
     </td>
   </tr>
   <tr>
@@ -613,7 +618,8 @@ file://path/to/file (where file contains
         </li>
 
         <li> a path to a file containing either one of the above options </li>
-<pre><code> --master=file://path/to/file (where file contains one of the above)</code></pre>
+        <pre><code>You can also use the <code>file:///path/to/file</code>
syntax
+        to read the argument from a file which contains one of the above</code></pre>
         </li>
       </ol>
       Examples:
@@ -724,13 +730,18 @@ file://path/to/file (where file contains
       --credential=VALUE
     </td>
     <td>
-      Either a path to a text with a single line
+      A path to a text file with a single line
       containing 'principal' and 'secret' separated by whitespace.
+
       <p/>
       Or a path containing the JSON formatted information used for one credential.
       <p/>
-      Path could be of the form <code>file:///path/to/file< code> or <code>/path/to/file</code>.
+      Path should be of the form <code>file://path/to/file</code>.
       <p/>
+      Remember you can also use
+      the <code>file:///path/to/file</code> argument value format to read the
+      value from a file.
+      </p>
       JSON file example:
 <pre><code>{
   "principal": "username",
@@ -930,9 +941,8 @@ file://path/to/file (where file contains
       List of modules to be loaded and be available to the internal
       subsystems.
       <p/>
-      Use <code>--modules=filepath</code> to specify the list of modules via
a
-      file containing a JSON formatted string. 'filepath' can be
-      of the form <code>file:///path/to/file</code> or <code>/path/to/file</code>.
+      Remember you can also use the <code>file:///path/to/file</code>
+      argument value format to have the value read from a file.
       <p/>
       Use <code>--modules="{...}"</code> to specify the list of modules inline.
       <p/>

Modified: mesos/site/source/documentation/latest/getting-started.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/getting-started.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/getting-started.md (original)
+++ mesos/site/source/documentation/latest/getting-started.md Tue Feb 24 05:22:07 2015
@@ -13,9 +13,9 @@ There are different ways you can get Mes
         $ wget http://www.apache.org/dist/mesos/0.20.1/mesos-0.20.1.tar.gz
         $ tar -zxf mesos-0.20.1.tar.gz
 
-2. Clone the Mesos git [repository](http://git-wip-us.apache.org/repos/asf/mesos.git) (***Advanced
Users Only***)
+2. Clone the Mesos git [repository](https://git-wip-us.apache.org/repos/asf/mesos.git) (***Advanced
Users Only***)
 
-        $ git clone http://git-wip-us.apache.org/repos/asf/mesos.git
+        $ git clone https://git-wip-us.apache.org/repos/asf/mesos.git
 
 ## System Requirements
 
@@ -77,7 +77,7 @@ There are different ways you can get Mes
 
         $ sudo yum groupinstall -y "Development Tools"
 
-        $ sudo yum install -y python-devel java-1.7.0-openjdk-devel zlib-devel libcurl-devel
openssl-devel cyrus-sasl-devel cyrus-sasl-md5 apr-devel subversion-devel
+        $ sudo yum install -y python-devel java-1.7.0-openjdk-devel zlib-devel libcurl-devel
openssl-devel cyrus-sasl-devel cyrus-sasl-md5 apr-devel subversion-devel apr-utils-devel
 
         # Install maven.
         $ wget http://mirror.nexcess.net/apache/maven/maven-3/3.0.5/binaries/apache-maven-3.0.5-bin.tar.gz

Modified: mesos/site/source/documentation/latest/mesos-c++-style-guide.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/mesos-c%2B%2B-style-guide.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/mesos-c++-style-guide.md (original)
+++ mesos/site/source/documentation/latest/mesos-c++-style-guide.md Tue Feb 24 05:22:07 2015
@@ -103,7 +103,7 @@ const typename map::iterator& i = values
 // 2: Don't use.
 auto authorizer = LocalAuthorizer::create(acls);
 // Compare with
-Try<Owned<LocalAuthorizer>> authorizer = LocalAuthorizer::create();
+Try&lt;Owned&lt;LocalAuthorizer>> authorizer = LocalAuthorizer::create();
 </pre>
 
 * Rvalue references.

Modified: mesos/site/source/documentation/latest/modules.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/modules.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/modules.md (original)
+++ mesos/site/source/documentation/latest/modules.md Tue Feb 24 05:22:07 2015
@@ -8,9 +8,7 @@ Experimental support for Mesos modules w
 
 ### Disclaimer
 
-- Use and development of Mesos modules is at own risk! Only graced modules
-(modules that are part of Mesos distribution) are maintained by the Mesos
-project.
+- Use and development of Mesos modules is at own risk!
 
 - Please direct all questions to the relevant module writer and/or write to
 modules@mesos.apache.org. Questions related to modules sent to user and dev list
@@ -119,8 +117,39 @@ If both "file" and "name" are specified,
 
 ## What kinds of modules are supported?
 
-Mesos currently only provides Isolator and Authentication modules.  Additional
-graced modules will be added in the near future.
+Here are the various module kinds currently available:
+
+### Anonymous
+
+Anonymous modules do __not__ receive any callbacks but simply
+coexists with their parent process.
+
+Unlike other named modules, an anonymous module does not directly
+replace or provide essential Mesos functionality (such as an
+Isolator module does). Unlike a decorator module it does not
+directly add or inject data into Mesos core either.
+
+Anonymous modules do not require any specific selectors (flags) as
+they are immediately instantiated when getting loaded via the
+`--modules` flag by the Mesos master or slave.
+
+### Authentication
+
+Authenticatee and Authenticator modules allow for third parties to quickly
+develop and plug-in new authentication methods. An example for such modules
+could be to support PAM (LDAP, MySQL, NIS, UNIX) backed authentication.
+
+### Hook
+
+Similar to Apache Webserver Modules, hooks allows module writers to tie into
+internal components which may not be suitable to be abstracted entirely behind
+modules but rather let's them define actions on so-called hooks.
+
+### Isolator
+
+Isolator modules enable experimenting with specialized isolation and monitoring
+capabilities. Examples of these could be 3rdparty resource isolation mechanisms
+for GPGPU hardware, networking, etc.
 
 ## Writing Mesos modules
 

Modified: mesos/site/source/documentation/latest/release-guide.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/release-guide.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/release-guide.md (original)
+++ mesos/site/source/documentation/latest/release-guide.md Tue Feb 24 05:22:07 2015
@@ -138,3 +138,7 @@ Per the guidelines [when to archive](htt
 1. Checkout the mesos distribution folder: `svn co https://dist.apache.org/repos/dist/release/mesos`
 
 2. Remove all minor versions that are no longer under development and commit the change.
+
+## Set the release date
+
+1. Find the released Mesos version on https://issues.apache.org/jira/plugins/servlet/project-config/MESOS/versions,
and update the release date.

Modified: mesos/site/source/documentation/latest/slave-recovery.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/slave-recovery.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/slave-recovery.md (original)
+++ mesos/site/source/documentation/latest/slave-recovery.md Tue Feb 24 05:22:07 2015
@@ -15,18 +15,20 @@ Mesos slave could be restarted for an up
 
 Slave recovery works by having the slave checkpoint enough information (e.g., Task Info,
Executor Info, Status Updates) about the running tasks and executors to local disk. Once the
slave ***and*** the framework(s) enable checkpointing, any subsequent slave restarts would
recover the checkpointed information and reconnect with the executors. Note that if the host
running the slave process is rebooted all the executors/tasks are killed.
 
-> NOTE: To enable slave recovery both the slave and the framework should explicitly request
checkpointing.
+> NOTE: To enable recovery the framework should explicitly request checkpointing.
 > Alternatively, a framework that doesn't want the disk i/o overhead of checkpointing
can opt out of checkpointing.
 
 
 ## Enabling slave checkpointing
+> NOTE: From Mesos 0.22.0 slave checkpointing will be automatically enabled for all slaves.
 
 As part of this feature, 4 new flags were added to the slave.
 
 * `checkpoint` :  Whether to checkpoint slave and frameworks information
-                  to disk [Default: false].
+                  to disk [Default: true].
     - This enables a restarted slave to recover status updates and reconnect
       with (--recover=reconnect) or kill (--recover=kill) old executors.
+    > NOTE: From Mesos 0.22.0 this flag will be removed as it will be enabled for all
slaves.
 
 	* `strict` : Whether to do recovery in strict mode [Default: true].
 		* If strict=true, any and all recovery errors are considered fatal.
@@ -45,7 +47,7 @@ As part of this feature, 4 new flags wer
     reconnect to the slave will self-terminate.
     NOTE: This flag is only applicable when `--checkpoint` is enabled.
 
-> NOTE: If checkpointing is enabled on the slave, but none of the frameworks have enabled
checkpointing,
+> NOTE: If none of the frameworks have enabled checkpointing,
 > executors/tasks of frameworks die when the slave dies and are not recovered.
 
 A restarted slave should re-register with master within a timeout (currently, 75s). If the
slave takes longer

Modified: mesos/site/source/documentation/latest/upgrades.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/upgrades.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/upgrades.md (original)
+++ mesos/site/source/documentation/latest/upgrades.md Tue Feb 24 05:22:07 2015
@@ -8,6 +8,8 @@ This document serves as a guide for user
 
 ## (WIP) Upgrading from 0.21.x to 0.22.x
 
+**NOTE** The C++/Java/Python scheduler bindings have been updated. In particular, the driver
can be constructed with an additional argument that specifies whether to use implicit driver
acknowledgements. In `statusUpdate`, the `TaskStatus` now includes a UUID to make explicit
acknowledgements possible.
+
 **NOTE**: The Authentication API has changed slightly in this release to support additional
authentication mechanisms. The change from 'string' to 'bytes' for AuthenticationStartMessage.data
has no impact on C++ or the over-the-wire representation, so it only impacts pure language
bindings for languages like Java and Python that use different types for UTF-8 strings vs.
byte arrays.
 
 ```
@@ -17,6 +19,17 @@ message AuthenticationStartMessage {
 }
 ```
 
+**NOTE** All Mesos arguments can now be passed using file:// to read them out of a file (either
an absolute or relative path). The --credentials, --whitelist, and any flags that expect JSON
backed arguments (such as --modules) behave as before, although support for just passing a
absolute path for any JSON flags rather than file:// has been deprecated and will produce
a warning (and the absolute path behavior will be removed in a future release).
+
+In order to upgrade a running cluster:
+
+* Install the new master binaries and restart the masters.
+* Install the new slave binaries and restart the slaves.
+* Upgrade the schedulers:
+  * For Java schedulers, link the new native library against the new JAR. The JAR contains
API changes per the **NOTE** above. A 0.21.0 JAR will work with a 0.22.0 libmesos. A 0.22.0
JAR will work with a 0.21.0 libmesos if explicit acks are not being used. 0.22.0 and 0.21.0
are inter-operable at the protocol level between the master and the scheduler.
+  * For Python schedulers, upgrade to use a 0.22.0 egg. If constructing `MesosSchedulerDriverImpl`
with `Credentials`, your code must be updated to pass the `implicitAcknowledgements` argument
before `Credentials`. You may run a 0.21.0 Python scheduler against a 0.22.0 master, and vice
versa.
+* Restart the schedulers.
+* Upgrade the executors by linking the latest native library / jar / egg.
 
 ## Upgrading from 0.20.x to 0.21.x
 



Mime
View raw message