Return-Path: X-Original-To: apmail-aurora-commits-archive@minotaur.apache.org Delivered-To: apmail-aurora-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6B009172E0 for ; Fri, 9 Jan 2015 21:17:40 +0000 (UTC) Received: (qmail 83145 invoked by uid 500); 9 Jan 2015 21:17:41 -0000 Delivered-To: apmail-aurora-commits-archive@aurora.apache.org Received: (qmail 83109 invoked by uid 500); 9 Jan 2015 21:17:41 -0000 Mailing-List: contact commits-help@aurora.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@aurora.incubator.apache.org Delivered-To: mailing list commits@aurora.incubator.apache.org Received: (qmail 83100 invoked by uid 99); 9 Jan 2015 21:17:41 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 09 Jan 2015 21:17:41 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED,T_RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Fri, 09 Jan 2015 21:17:39 +0000 Received: (qmail 80035 invoked by uid 99); 9 Jan 2015 21:17:19 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 09 Jan 2015 21:17:19 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id 298458173E8; Fri, 9 Jan 2015 21:17:19 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: wfarner@apache.org To: commits@aurora.incubator.apache.org Message-Id: <1192189be8db4377b4180eb14b3bdd2a@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: incubator-aurora git commit: Scrub docs of remaining references to aurora2 and aurora help. Date: Fri, 9 Jan 2015 21:17:19 +0000 (UTC) X-Virus-Checked: Checked by ClamAV on apache.org Repository: incubator-aurora Updated Branches: refs/heads/master 8c2786437 -> 1f907da2c Scrub docs of remaining references to aurora2 and aurora help. Reviewed at https://reviews.apache.org/r/29774/ Project: http://git-wip-us.apache.org/repos/asf/incubator-aurora/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-aurora/commit/1f907da2 Tree: http://git-wip-us.apache.org/repos/asf/incubator-aurora/tree/1f907da2 Diff: http://git-wip-us.apache.org/repos/asf/incubator-aurora/diff/1f907da2 Branch: refs/heads/master Commit: 1f907da2c0d7a572d7e9091f5a303318b0692310 Parents: 8c27864 Author: Bill Farner Authored: Fri Jan 9 13:16:41 2015 -0800 Committer: Bill Farner Committed: Fri Jan 9 13:16:41 2015 -0800 ---------------------------------------------------------------------- docs/cron-jobs.md | 8 +- docs/developing-aurora-client.md | 87 +++------------------- docs/tutorial.md | 2 +- docs/user-guide.md | 12 +-- src/test/sh/org/apache/aurora/e2e/test_run.sh | 4 +- 5 files changed, 20 insertions(+), 93 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/1f907da2/docs/cron-jobs.md ---------------------------------------------------------------------- diff --git a/docs/cron-jobs.md b/docs/cron-jobs.md index c3e8eed..41770fd 100644 --- a/docs/cron-jobs.md +++ b/docs/cron-jobs.md @@ -72,24 +72,24 @@ set `max_task_failures` to `-1`. ## Interacting with cron jobs via the Aurora CLI -Most interaction with cron jobs takes place using the `cron` subcommand. See `aurora2 help cron` +Most interaction with cron jobs takes place using the `cron` subcommand. See `aurora cron -h` for up-to-date usage instructions. ### cron schedule Schedules a new cron job on the Aurora cluster for later runs or replaces the existing cron template with a new one. Only future runs will be affected, any existing active tasks are left intact. - $ aurora2 cron schedule devcluster/www-data/test/cron_hello_world /vagrant/examples/jobs/cron_hello_world.aurora + $ aurora cron schedule devcluster/www-data/test/cron_hello_world /vagrant/examples/jobs/cron_hello_world.aurora ### cron deschedule Deschedules a cron job, preventing future runs but allowing current runs to complete. - $ aurora2 cron deschedule devcluster/www-data/test/cron_hello_world + $ aurora cron deschedule devcluster/www-data/test/cron_hello_world ### cron start Start a cron job immediately, outside of its normal cron schedule. - $ aurora2 cron start devcluster/www-data/test/cron_hello_world + $ aurora cron start devcluster/www-data/test/cron_hello_world ### job killall, job restart, job kill Cron jobs create instances running on the cluster that you can interact with like normal Aurora http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/1f907da2/docs/developing-aurora-client.md ---------------------------------------------------------------------- diff --git a/docs/developing-aurora-client.md b/docs/developing-aurora-client.md index 83b843e..a7253d2 100644 --- a/docs/developing-aurora-client.md +++ b/docs/developing-aurora-client.md @@ -1,26 +1,8 @@ Getting Started =============== -Aurora consists of four main pieces: the scheduler (which finds resources in the cluster that can be used to run a job), the executor (which uses the resources assigned by the scheduler to run a job), the command-line client, and the web-ui. For information about working on the scheduler or the webUI, see [Developing the Aurora Scheduler](developing-aurora-scheduler.md). - -If you want to work on the command-line client, this is the place for you! - -The client is written in Python, and unlike the server side of things, we build the client using the Pants build tool, instead of Gradle. Pants is a tool that was built by twitter for handling builds of large collaborative systems. You can see a detailed explanation of -pants [here](http://pantsbuild.github.io/python-readme.html). - -To build the client executable, run the following in a command-shell: - - $ ./pants src/main/python/apache/aurora/client/cli:aurora - -This will produce a python executable _pex_ file in `dist/aurora.pex`. Pex files -are fully self-contained executables: just copy the pex file into your path, and you'll be able to run it. For example, for a typical installation: - - $ cp dist/aurora.pex /usr/local/bin/aurora - -To run all of the client tests: - - $ ./pants src/test/python/apache/aurora/client/:all - +The client is written in Python, and uses the +[Pants](http://pantsbuild.github.io/python-readme.html) build tool. Client Configuration ==================== @@ -34,69 +16,22 @@ how the client locates this file can be found in the Building and Testing the Client =============================== -Building and testing the client code are both done using Pants. The relevant targets to know about are: +Building and testing the client code are both done using Pants. The relevant targets to know about +are: * Build a client executable: `./pants src/main/python/apache/aurora/client/cli:aurora` * Test client code: `./pants ./pants src/test/python/apache/aurora/client/cli:all` -Overview of the Client Architecture -=================================== - -The client is built on a stacked architecture: - - 1. At the lowest level, we have a thrift RPC API interface - to the aurora scheduler. The interface is declared in thrift, in the file - `src/main/thrift/org/apache/aurora/gen/api.thrift`. - - 2. On top of the primitive API, we have a client API. The client API - takes the primitive operations provided by the scheduler, and uses them - to implement client-side behaviors. For example, when you update a job, - on the scheduler, that's done by a sequence of operations. The sequence is implemented - by the client API `update` method, which does the following using the thrift API: - * fetching the state of task instances in the mesos cluster, and figuring out which need - to be updated; - * For each task to be updated: - - killing the old version; - - starting the new version; - - monitoring the new version to ensure that the update succeeded. - 3. On top of the API, we have the command-line client itself. The core client, at this level, - consists of the interface to the command-line which the user will use to interact with aurora. - The client code is found in `src/python/apache/aurora/client/cli`. In the `cli` directory, - the rough structure is as follows: - * `__init__.py` contains the noun/verb command-line processing framework used by client. - * `jobs.py` contains the implementation of the core `job` noun, and all of its operations. - * `client.py` contains the code that binds the client nouns and verbs into an executable. - Running/Debugging the Client ============================ -For manually testing client changes against a cluster, we use vagrant. To start a virtual cluster, -you need to install a working vagrant environment, and then run "vagrant up" for the root of -the aurora workspace. This will create a vagrant host named "devcluster", with a mesos master, -a set of mesos slaves, and an aurora scheduler. - -To use the devcluster, you need to bring it up by running `vagrant up`, and then connect to the vagrant host using `vagrant ssh`. This will open a bash session on the virtual machine hosting the devcluster. In the home directory, there are two key paths to know about: - - * `~/aurora`: this is a copy of the git workspace in which you launched the vagrant cluster. - To test client changes, you'll use this copy. - * `/vagrant`: this is a mounted filesystem that's a direct image of your git workspace. - This isn't a copy - it is your git workspace. Editing files on your host machine will - be immediately visible here, because they are the same files. - -Whenever the scheduler is modified, to update your vagrant environment to use the new scheduler, -you'll need to re-initialize your vagrant images. To do this, you need to run two commands: - - * `vagrant destroy`: this will delete the old devcluster image. - * `vagrant up`: this creates a fresh devcluster image based on the current state of your workspace. +For manually testing client changes against a cluster, we use [Vagrant](https://www.vagrantup.com/). +To start a virtual cluster, you need to install Vagrant, and then run `vagrant up` for the root of +the aurora workspace. This will create a vagrant host named "devcluster", with a mesos master, a set +of mesos slaves, and an aurora scheduler. -You should try to minimize rebuilding vagrant images; it's not horribly slow, but it does take a while. +If you have changed you would like to test in your local cluster, you'll rebuild the client: -To test client changes: + vagrant ssh -c 'aurorabuild client' - * Make a change in your local workspace, and commit it. - * `vagrant ssh` into the devcluster. - * `cd aurora` - * Pull your changes into the vagrant copy: `git pull /vagrant *branchname*`. - * Build the modified client using pants. - * Run your command using `aurora2`. (You don't need to do any install; the aurora2 command - is a symbolic link to the executable generated by pants.) +Once this completes, the `aurora` command will reflect your changes. http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/1f907da2/docs/tutorial.md ---------------------------------------------------------------------- diff --git a/docs/tutorial.md b/docs/tutorial.md index e46addd..7ea36e4 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -261,5 +261,5 @@ Now that you've finished this Tutorial, you should read or do the following: the [Aurora + Thermos Configuration Reference](configuration-reference.md). - The [Aurora User Guide](user-guide.md) provides an overview of how Aurora, Mesos, and Thermos work "under the hood". -- Explore the Aurora Client - use the `aurora help` subcommand, and read the +- Explore the Aurora Client - use `aurora -h`, and read the [Aurora Client Commands](client-commands.md) document. http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/1f907da2/docs/user-guide.md ---------------------------------------------------------------------- diff --git a/docs/user-guide.md b/docs/user-guide.md index 877f5b7..154b470 100644 --- a/docs/user-guide.md +++ b/docs/user-guide.md @@ -350,14 +350,6 @@ You interact with Aurora jobs either via: jobs, and finished jobs. Jobs are arranged by role, typically a service account for production jobs and user accounts for test or development jobs. -- The Aurora Client's command line interface +- The Aurora client - Several Client commands have a `-o` option that automatically opens a window to - the specified Job's scheduler UI URL. And, as described above, the `open` command also takes - you there. - - For a complete list of Aurora Client commands, use `aurora help` and, for specific - command help, `aurora help [command]`. **Note**: `aurora help open` - returns `"subcommand open not found"` due to our reflection tricks not - working on words that are also builtin Python function names. Or see the - [Aurora Client Commands](client-commands.md) document. + See [client commands](client-commands.md). http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/1f907da2/src/test/sh/org/apache/aurora/e2e/test_run.sh ---------------------------------------------------------------------- diff --git a/src/test/sh/org/apache/aurora/e2e/test_run.sh b/src/test/sh/org/apache/aurora/e2e/test_run.sh index c3822ad..ab91b95 100755 --- a/src/test/sh/org/apache/aurora/e2e/test_run.sh +++ b/src/test/sh/org/apache/aurora/e2e/test_run.sh @@ -13,14 +13,14 @@ # limitations under the License. # # -# Test aurora run/aurora2 task run. This script is meant to be run within an ssh-agent to +# Test aurora run/aurora task run. This script is meant to be run within an ssh-agent to # avoid polluting the global ssh-agent w/ the vagrant private key. set -u -e -x jobkey=$1 scheduler_ip=$2 -aurora_command=${3:-"aurora2 task run"} +aurora_command=${3:-"aurora task run"} ssh-add $HOME/.vagrant.d/insecure_private_key vagrant ssh -c "$aurora_command $jobkey 'pwd'" -- -A -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \