aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Chu-Carroll" <mchucarr...@twopensource.com>
Subject Re: Review Request 17457: Initial version of an end-to-end test using vagrant
Date Mon, 03 Feb 2014 22:55:07 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17457/
-----------------------------------------------------------

(Updated Feb. 3, 2014, 5:55 p.m.)


Review request for Aurora, Bill Farner and Brian Wickman.


Changes
-------

Addressed more review comments.


Bugs: aurora-125 and aurora-80
    https://issues.apache.org/jira/browse/aurora-125
    https://issues.apache.org/jira/browse/aurora-80


Repository: aurora


Description
-------

Got an initial version of the end-to-end test working. 

Currently creates a job, probes it via the scheduler pages,  updates it, and kills it. 
AURORA-127 will extend this to run multiple instances; AURORA-125 will extend this to use
"aurora run" and "aurora ssh" to probe the running tasks.


Diffs (updated)
-----

  3rdparty/python/BUILD cc3e8c929e7b2b614a6ea3dbbb4e70d2f8d22586 
  Vagrantfile 98dcf439aed53da5e78440de04562baf63c7272f 
  examples/vagrant/provision-aurora-scheduler.sh e48291a8673aeb8888169747486b7d50a54f5a56

  examples/vagrant/provision-dev-environment.sh 7fc11bf523a054f6d068cf676e87bb043d854a25 
  examples/vagrant/provision-mesos-slave.sh c2548a268e0fad7aa4f84d3fa48aea012007502b 
  src/test/sh/org/apache/aurora/e2e/flask/BUILD PRE-CREATION 
  src/test/sh/org/apache/aurora/e2e/flask/flask_example.aurora PRE-CREATION 
  src/test/sh/org/apache/aurora/e2e/flask/flask_example.py PRE-CREATION 
  src/test/sh/org/apache/aurora/e2e/flask/flask_example_updated.aurora PRE-CREATION 
  src/test/sh/org/apache/aurora/e2e/test_common.sh PRE-CREATION 
  src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/17457/diff/


Testing
-------

[sun-wukong incubator-aurora (e2e)]$ sh src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
Bringing machine 'devtools' up with 'virtualbox' provider...
Bringing machine 'zookeeper' up with 'virtualbox' provider...
Bringing machine 'mesos-master' up with 'virtualbox' provider...
Bringing machine 'mesos-slave' up with 'virtualbox' provider...
Bringing machine 'aurora-scheduler' up with 'virtualbox' provider...
[devtools] VirtualBox VM is already running.
[zookeeper] VirtualBox VM is already running.
[mesos-master] VirtualBox VM is already running.
[mesos-slave] VirtualBox VM is already running.
[aurora-scheduler] VirtualBox VM is already running.
Cloning into '/home/vagrant/test_dev'...
done.
Already on 'e2e'
Already up-to-date.
+ VIRTUALENV_VERSION=1.10.1
+ which python2.7
++ which python2.7
+ PY=/usr/bin/python2.7
+ echo 'Using /usr/bin/python2.7'
Using /usr/bin/python2.7
+ '[' -f build-support/virtualenv-1.10.1/BOOTSTRAPPED ']'
+ pushd build-support
~/test_dev/build-support ~/test_dev
+ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.10.1.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1294k  100 1294k    0     0  3261k      0 --:--:-- --:--:-- --:--:-- 3817k
+ tar zxvf virtualenv-1.10.1.tar.gz
virtualenv-1.10.1/
virtualenv-1.10.1/AUTHORS.txt
virtualenv-1.10.1/bin/
virtualenv-1.10.1/bin/rebuild-script.py
virtualenv-1.10.1/docs/
virtualenv-1.10.1/docs/_theme/
virtualenv-1.10.1/docs/_theme/nature/
virtualenv-1.10.1/docs/_theme/nature/static/
virtualenv-1.10.1/docs/_theme/nature/static/nature.css_t
virtualenv-1.10.1/docs/_theme/nature/static/pygments.css
virtualenv-1.10.1/docs/_theme/nature/theme.conf
virtualenv-1.10.1/docs/conf.py
virtualenv-1.10.1/docs/index.rst
virtualenv-1.10.1/docs/make.bat
virtualenv-1.10.1/docs/Makefile
virtualenv-1.10.1/docs/news.rst
virtualenv-1.10.1/LICENSE.txt
virtualenv-1.10.1/MANIFEST.in
virtualenv-1.10.1/PKG-INFO
virtualenv-1.10.1/README.rst
virtualenv-1.10.1/scripts/
virtualenv-1.10.1/scripts/virtualenv
virtualenv-1.10.1/setup.cfg
virtualenv-1.10.1/setup.py
virtualenv-1.10.1/virtualenv.egg-info/
virtualenv-1.10.1/virtualenv.egg-info/dependency_links.txt
virtualenv-1.10.1/virtualenv.egg-info/entry_points.txt
virtualenv-1.10.1/virtualenv.egg-info/not-zip-safe
virtualenv-1.10.1/virtualenv.egg-info/PKG-INFO
virtualenv-1.10.1/virtualenv.egg-info/SOURCES.txt
virtualenv-1.10.1/virtualenv.egg-info/top_level.txt
virtualenv-1.10.1/virtualenv.py
virtualenv-1.10.1/virtualenv_embedded/
virtualenv-1.10.1/virtualenv_embedded/activate.bat
virtualenv-1.10.1/virtualenv_embedded/activate.csh
virtualenv-1.10.1/virtualenv_embedded/activate.fish
virtualenv-1.10.1/virtualenv_embedded/activate.ps1
virtualenv-1.10.1/virtualenv_embedded/activate.sh
virtualenv-1.10.1/virtualenv_embedded/activate_this.py
virtualenv-1.10.1/virtualenv_embedded/deactivate.bat
virtualenv-1.10.1/virtualenv_embedded/distutils-init.py
virtualenv-1.10.1/virtualenv_embedded/distutils.cfg
virtualenv-1.10.1/virtualenv_embedded/site.py
virtualenv-1.10.1/virtualenv_support/
virtualenv-1.10.1/virtualenv_support/__init__.py
virtualenv-1.10.1/virtualenv_support/pip-1.4.1.tar.gz
virtualenv-1.10.1/virtualenv_support/setuptools-0.9.8.tar.gz
+ touch virtualenv-1.10.1/BOOTSTRAPPED
~/test_dev
+ popd
+ exec /usr/bin/python2.7 build-support/virtualenv-1.10.1/virtualenv.py /tmp/pants.BqUkIw/tcp
New python executable in /tmp/pants.BqUkIw/tcp/bin/python2.7
Also creating executable in /tmp/pants.BqUkIw/tcp/bin/python
Installing Setuptools..............................................................................................................................................................................................................................done.
Installing Pip.....................................................................................................................................................................................................................................................................................................................................done.
/tmp/pants.BqUkIw ~/test_dev
Downloading/unpacking twitter.common.python==0.2.0
  Running setup.py egg_info for package twitter.common.python

Downloading/unpacking setuptools==1.1.7 (from twitter.common.python==0.2.0)
  Running setup.py egg_info for package setuptools

Installing collected packages: twitter.common.python, setuptools
  Running setup.py install for twitter.common.python
    Skipping installation of /tmp/pants.BqUkIw/tcp/lib/python2.7/site-packages/twitter/__init__.py
(namespace package)
    Skipping installation of /tmp/pants.BqUkIw/tcp/lib/python2.7/site-packages/twitter/common/__init__.py
(namespace package)

    Installing /tmp/pants.BqUkIw/tcp/lib/python2.7/site-packages/twitter.common.python-0.2.0-py2.7-nspkg.pth
  Found existing installation: setuptools 0.9.8
    Uninstalling setuptools:
      Successfully uninstalled setuptools
  Running setup.py install for setuptools

    Installing easy_install script to /tmp/pants.BqUkIw/tcp/bin
    Installing easy_install-2.7 script to /tmp/pants.BqUkIw/tcp/bin
Successfully installed twitter.common.python setuptools
Cleaning up...
twitter.common.python: Obtaining twitter.pants==0.0.13
twitter.common.python: Obtaining twitter.pants==0.0.13 :: Installing twitter.pants
twitter.common.python: Obtaining twitter.pants==0.0.13 :: Distilling twitter.pants
twitter.common.python: Obtaining twitter.pants==0.0.13: 1413.4ms
twitter.common.python:   Installing twitter.pants: 0.4ms
twitter.common.python:   Distilling twitter.pants: 798.7ms
twitter.common.python: Obtaining twitter.common.quantity==0.1.0
twitter.common.python: Obtaining twitter.common.quantity==0.1.0 :: Installing twitter.common.quantity
twitter.common.python: Obtaining twitter.common.quantity==0.1.0 :: Distilling twitter.common.quantity
twitter.common.python: Obtaining twitter.common.quantity==0.1.0: 646.7ms
twitter.common.python:   Installing twitter.common.quantity: 0.2ms
twitter.common.python:   Distilling twitter.common.quantity: 94.6ms
twitter.common.python: Obtaining elementtree
twitter.common.python: Obtaining elementtree :: Installing elementtree
twitter.common.python: Obtaining elementtree :: Distilling elementtree
twitter.common.python: Obtaining elementtree: 1681.4ms
twitter.common.python:   Installing elementtree: 0.2ms
twitter.common.python:   Distilling elementtree: 140.7ms
twitter.common.python: Obtaining twitter.common.config==0.1.0
twitter.common.python: Obtaining twitter.common.config==0.1.0 :: Installing twitter.common.config
twitter.common.python: Obtaining twitter.common.config==0.1.0 :: Distilling twitter.common.config
twitter.common.python: Obtaining twitter.common.config==0.1.0: 490.8ms
twitter.common.python:   Installing twitter.common.config: 0.2ms
twitter.common.python:   Distilling twitter.common.config: 92.8ms
twitter.common.python: Obtaining ansicolors
twitter.common.python: Obtaining ansicolors :: Installing ansicolors
twitter.common.python: Obtaining ansicolors :: Distilling ansicolors
twitter.common.python: Obtaining ansicolors: 603.0ms
twitter.common.python:   Installing ansicolors: 0.2ms
twitter.common.python:   Distilling ansicolors: 86.1ms
twitter.common.python: Obtaining twitter.common.confluence==0.1.0
twitter.common.python: Obtaining twitter.common.confluence==0.1.0 :: Installing twitter.common.confluence
twitter.common.python: Obtaining twitter.common.confluence==0.1.0 :: Distilling twitter.common.confluence
twitter.common.python: Obtaining twitter.common.confluence==0.1.0: 759.4ms
twitter.common.python:   Installing twitter.common.confluence: 0.3ms
twitter.common.python:   Distilling twitter.common.confluence: 95.8ms
twitter.common.python: Obtaining psutil==1.1.2
twitter.common.python: Obtaining psutil==1.1.2 :: Installing psutil
twitter.common.python: Obtaining psutil==1.1.2 :: Distilling psutil
twitter.common.python: Obtaining psutil==1.1.2: 1487.2ms
twitter.common.python:   Installing psutil: 0.4ms
twitter.common.python:   Distilling psutil: 446.3ms
twitter.common.python: Obtaining pygments
twitter.common.python: Obtaining pygments: 761.2ms
twitter.common.python: Obtaining twitter.common.decorators==0.1.0
twitter.common.python: Obtaining twitter.common.decorators==0.1.0 :: Installing twitter.common.decorators
twitter.common.python: Obtaining twitter.common.decorators==0.1.0 :: Distilling twitter.common.decorators
twitter.common.python: Obtaining twitter.common.decorators==0.1.0: 582.7ms
twitter.common.python:   Installing twitter.common.decorators: 0.2ms
twitter.common.python:   Distilling twitter.common.decorators: 94.8ms
twitter.common.python: Obtaining pytest
twitter.common.python: Obtaining pytest :: Installing pytest
twitter.common.python: Obtaining pytest :: Distilling pytest
twitter.common.python: Obtaining pytest: 1700.3ms
twitter.common.python:   Installing pytest: 0.3ms
twitter.common.python:   Distilling pytest: 413.1ms
twitter.common.python: Obtaining pylint
twitter.common.python: Obtaining pylint :: Installing pylint
twitter.common.python: Obtaining pylint :: Distilling pylint
twitter.common.python: Obtaining pylint: 1301.7ms
twitter.common.python:   Installing pylint: 0.3ms
twitter.common.python:   Distilling pylint: 440.7ms
twitter.common.python: Obtaining pytest-cov
twitter.common.python: Obtaining pytest-cov :: Installing pytest-cov
twitter.common.python: Obtaining pytest-cov :: Distilling pytest-cov
twitter.common.python: Obtaining pytest-cov: 532.5ms
twitter.common.python:   Installing pytest-cov: 0.2ms
twitter.common.python:   Distilling pytest-cov: 88.1ms
twitter.common.python: Obtaining twitter.common.process==0.1.0
twitter.common.python: Obtaining twitter.common.process==0.1.0 :: Installing twitter.common.process
twitter.common.python: Obtaining twitter.common.process==0.1.0 :: Distilling twitter.common.process
twitter.common.python: Obtaining twitter.common.process==0.1.0: 519.6ms
twitter.common.python:   Installing twitter.common.process: 0.2ms
twitter.common.python:   Distilling twitter.common.process: 110.1ms
twitter.common.python: Obtaining twitter.common.util==0.1.0
twitter.common.python: Obtaining twitter.common.util==0.1.0 :: Installing twitter.common.util
twitter.common.python: Obtaining twitter.common.util==0.1.0 :: Distilling twitter.common.util
twitter.common.python: Obtaining twitter.common.util==0.1.0: 520.6ms
twitter.common.python:   Installing twitter.common.util: 0.2ms
twitter.common.python:   Distilling twitter.common.util: 91.9ms
twitter.common.python: Obtaining markdown
twitter.common.python: Obtaining markdown :: Installing Markdown
twitter.common.python: Obtaining markdown :: Distilling Markdown
twitter.common.python: Obtaining markdown: 1247.1ms
twitter.common.python:   Installing Markdown: 0.5ms
twitter.common.python:   Distilling Markdown: 495.1ms
twitter.common.python: Obtaining twitter.common.python==0.2.2
twitter.common.python: Obtaining twitter.common.python==0.2.2 :: Installing twitter.common.python
twitter.common.python: Obtaining twitter.common.python==0.2.2 :: Distilling twitter.common.python
twitter.common.python: Obtaining twitter.common.python==0.2.2: 865.8ms
twitter.common.python:   Installing twitter.common.python: 0.3ms
twitter.common.python:   Distilling twitter.common.python: 238.7ms
twitter.common.python: Obtaining mako
twitter.common.python: Obtaining mako :: Installing Mako
twitter.common.python: Obtaining mako :: Distilling Mako
twitter.common.python: Obtaining mako: 1069.3ms
twitter.common.python:   Installing Mako: 0.3ms
twitter.common.python:   Distilling Mako: 291.9ms
twitter.common.python: Obtaining twitter.common.lang==0.1.0
twitter.common.python: Obtaining twitter.common.lang==0.1.0 :: Installing twitter.common.lang
twitter.common.python: Obtaining twitter.common.lang==0.1.0 :: Distilling twitter.common.lang
twitter.common.python: Obtaining twitter.common.lang==0.1.0: 796.2ms
twitter.common.python:   Installing twitter.common.lang: 0.2ms
twitter.common.python:   Distilling twitter.common.lang: 95.4ms
twitter.common.python: Obtaining twitter.common.collections==0.1.0
twitter.common.python: Obtaining twitter.common.collections==0.1.0 :: Installing twitter.common.collections
twitter.common.python: Obtaining twitter.common.collections==0.1.0 :: Distilling twitter.common.collections
twitter.common.python: Obtaining twitter.common.collections==0.1.0: 554.6ms
twitter.common.python:   Installing twitter.common.collections: 0.2ms
twitter.common.python:   Distilling twitter.common.collections: 103.2ms
twitter.common.python: Obtaining twitter.common.options==0.1.0
twitter.common.python: Obtaining twitter.common.options==0.1.0 :: Installing twitter.common.options
twitter.common.python: Obtaining twitter.common.options==0.1.0 :: Distilling twitter.common.options
twitter.common.python: Obtaining twitter.common.options==0.1.0: 614.5ms
twitter.common.python:   Installing twitter.common.options: 0.2ms
twitter.common.python:   Distilling twitter.common.options: 92.8ms
twitter.common.python: Obtaining twitter.common.dirutil==0.1.0
twitter.common.python: Obtaining twitter.common.dirutil==0.1.0 :: Installing twitter.common.dirutil
twitter.common.python: Obtaining twitter.common.dirutil==0.1.0 :: Distilling twitter.common.dirutil
twitter.common.python: Obtaining twitter.common.dirutil==0.1.0: 537.3ms
twitter.common.python:   Installing twitter.common.dirutil: 0.4ms
twitter.common.python:   Distilling twitter.common.dirutil: 109.1ms
twitter.common.python: Obtaining twitter.common.log==0.1.0
twitter.common.python: Obtaining twitter.common.log==0.1.0 :: Installing twitter.common.log
twitter.common.python: Obtaining twitter.common.log==0.1.0 :: Distilling twitter.common.log
twitter.common.python: Obtaining twitter.common.log==0.1.0: 766.4ms
twitter.common.python:   Installing twitter.common.log: 0.2ms
twitter.common.python:   Distilling twitter.common.log: 135.2ms
twitter.common.python: Obtaining py>=1.4.19
twitter.common.python: Obtaining py>=1.4.19 :: Installing py
twitter.common.python: Obtaining py>=1.4.19 :: Distilling py
twitter.common.python: Obtaining py>=1.4.19: 871.5ms
twitter.common.python:   Installing py: 0.3ms
twitter.common.python:   Distilling py: 319.6ms
twitter.common.python: Obtaining astroid>=0.24.3
twitter.common.python: Obtaining astroid>=0.24.3 :: Installing astroid
twitter.common.python: Obtaining astroid>=0.24.3 :: Distilling astroid
twitter.common.python: Obtaining astroid>=0.24.3: 967.3ms
twitter.common.python:   Installing astroid: 0.4ms
twitter.common.python:   Distilling astroid: 399.9ms
twitter.common.python: Obtaining logilab-common>=0.53.0
twitter.common.python: Obtaining logilab-common>=0.53.0 :: Installing logilab-common
twitter.common.python: Obtaining logilab-common>=0.53.0 :: Distilling logilab-common
twitter.common.python: Obtaining logilab-common>=0.53.0: 1364.0ms
twitter.common.python:   Installing logilab-common: 0.4ms
twitter.common.python:   Distilling logilab-common: 603.8ms
twitter.common.python: Obtaining cov-core>=1.6
twitter.common.python: Obtaining cov-core>=1.6 :: Installing cov-core
twitter.common.python: Obtaining cov-core>=1.6 :: Distilling cov-core
twitter.common.python: Obtaining cov-core>=1.6: 471.0ms
twitter.common.python:   Installing cov-core: 0.4ms
twitter.common.python:   Distilling cov-core: 93.6ms
twitter.common.python: Obtaining twitter.common.string==0.1.0
twitter.common.python: Obtaining twitter.common.string==0.1.0 :: Installing twitter.common.string
twitter.common.python: Obtaining twitter.common.string==0.1.0 :: Distilling twitter.common.string
twitter.common.python: Obtaining twitter.common.string==0.1.0: 579.7ms
twitter.common.python:   Installing twitter.common.string: 0.2ms
twitter.common.python:   Distilling twitter.common.string: 93.8ms
twitter.common.python: Obtaining twitter.common.contextutil==0.1.0
twitter.common.python: Obtaining twitter.common.contextutil==0.1.0 :: Installing twitter.common.contextutil
twitter.common.python: Obtaining twitter.common.contextutil==0.1.0 :: Distilling twitter.common.contextutil
twitter.common.python: Obtaining twitter.common.contextutil==0.1.0: 524.2ms
twitter.common.python:   Installing twitter.common.contextutil: 0.5ms
twitter.common.python:   Distilling twitter.common.contextutil: 92.6ms
twitter.common.python: Obtaining setuptools==1.1.7
twitter.common.python: Obtaining setuptools==1.1.7 :: Installing setuptools
twitter.common.python: Obtaining setuptools==1.1.7 :: Distilling setuptools
twitter.common.python: Obtaining setuptools==1.1.7: 2212.2ms
twitter.common.python:   Installing setuptools: 0.3ms
twitter.common.python:   Distilling setuptools: 618.8ms
twitter.common.python: Obtaining MarkupSafe>=0.9.2
twitter.common.python: Obtaining MarkupSafe>=0.9.2 :: Installing MarkupSafe
twitter.common.python: Obtaining MarkupSafe>=0.9.2 :: Distilling MarkupSafe
twitter.common.python: Obtaining MarkupSafe>=0.9.2: 928.3ms
twitter.common.python:   Installing MarkupSafe: 0.3ms
twitter.common.python:   Distilling MarkupSafe: 251.4ms
twitter.common.python: Obtaining coverage>=3.4
twitter.common.python: Obtaining coverage>=3.4 :: Installing coverage
twitter.common.python: Obtaining coverage>=3.4 :: Distilling coverage
twitter.common.python: Obtaining coverage>=3.4: 1082.9ms
twitter.common.python:   Installing coverage: 0.3ms
twitter.common.python:   Distilling coverage: 425.6ms
~/test_dev
Build operating on targets: OrderedSet([PythonBinary(src/main/python/apache/aurora/client/bin/BUILD:aurora_client)])
Building PythonBinary PythonBinary(src/main/python/apache/aurora/client/bin/BUILD:aurora_client):
Wrote /home/vagrant/test_dev/dist/aurora_client.pex
Build operating on targets: OrderedSet([PythonBinary(src/test/sh/org/apache/aurora/e2e/flask/BUILD:flask_example)])
Building PythonBinary PythonBinary(src/test/sh/org/apache/aurora/e2e/flask/BUILD:flask_example):
Wrote /home/vagrant/test_dev/dist/flask_example.pex
== Creating job
 INFO] Creating job flask_example
 INFO] Response from scheduler: OK (message: 1 new tasks pending for job vagrant/test/flask_example)
 INFO] Job url: http://precise64:8081/scheduler/vagrant/test/flask_example
== Updating test job
 INFO] Updating job: flask_example
 INFO] Starting job update.
 INFO] Examining instances: [0]
 INFO] Killing instances: [0]
 INFO] Instances killed
 INFO] Adding instances: [0]
 INFO] Instances added
 INFO] Watching instances: [0]
 INFO] Detected RUNNING instance 0
 INFO] Instance 0 has been up and healthy for at least 30 seconds
 INFO] Update successful
 INFO] Response from scheduler: OK (message: Lock has been released.)
 INFO] Killing tasks for job: example/vagrant/test/flask_example
 INFO] Response from scheduler: OK (message: Tasks killed.)
 INFO] Job url: http://precise64:8081/scheduler/vagrant/test/flask_example
***
OK (all tests passed)
***
[sun-wukong incubator-aurora (e2e)]$


Thanks,

Mark Chu-Carroll


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message