aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Chu-Carroll" <mchucarr...@twopensource.com>
Subject Re: Review Request 19788: Add a clientv2 version of the e2e test.
Date Wed, 02 Apr 2014 01:00:48 GMT


> On March 28, 2014, 1:46 p.m., Maxim Khutornenko wrote:
> > src/test/sh/org/apache/aurora/e2e/test_end_to_end_v2.sh, line 43
> > <https://reviews.apache.org/r/19788/diff/1/?file=539838#file539838line43>
> >
> >     Why not pushd once instead?
> 
> Mark Chu-Carroll wrote:
>     Don't follow the question. What would a pushd accomplish here?
> 
> Maxim Khutornenko wrote:
>     Sorry, what I meant here ssh, pushd and do everything in a single session.

We get better errors this way. Right now, when the test setup fails, you can easily see exactly
which step was the problem. When you bundle it into one session, it becomes harder. The time-cost
of doing it this way is under 2 seconds, which is pretty much noise in the runtime of this
test.


- Mark


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19788/#review38919
-----------------------------------------------------------


On March 28, 2014, 1:28 p.m., Mark Chu-Carroll wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19788/
> -----------------------------------------------------------
> 
> (Updated March 28, 2014, 1:28 p.m.)
> 
> 
> Review request for Aurora, Joe Smith and Kevin Sweeney.
> 
> 
> Bugs: aurora-266
>     https://issues.apache.org/jira/browse/aurora-266
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> Add a clientv2 version of the e2e test.
> 
> 
> Diffs
> -----
> 
>   src/main/python/apache/aurora/client/BUILD 118c4a080b2aac88f1fb6365787b3bb55a665bea

>   src/main/python/apache/aurora/client/cli/BUILD 02e061c7d485cc7f24c48c0db1b5778819961f47

>   src/test/python/apache/aurora/client/cli/BUILD d93a2df0f146e2771de2ce6223438a671d0e92a2

>   src/test/sh/org/apache/aurora/e2e/test_end_to_end_v2.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/19788/diff/
> 
> 
> Testing
> -------
> 
> [sun-wukong incubator-aurora (v2_e2e)]$ sh src/test/sh/org/apache/aurora/e2e/test_end_to_end_v2.sh
> + . src/test/sh/org/apache/aurora/e2e/test_common.sh
> ++ exec
> ++ _aurora_built=false
> + RETCODE=1
> + export CLIENT_NAME=aurorav2.pex
> + CLIENT_NAME=aurorav2.pex
> + export EXAMPLE_DIR=/vagrant/src/test/sh/org/apache/aurora/e2e/flask
> + EXAMPLE_DIR=/vagrant/src/test/sh/org/apache/aurora/e2e/flask
> + TEST_DIR=deploy_test
> + TEST_CLUSTER=example
> + TEST_ROLE=vagrant
> + TEST_ENV=test
> + TEST_JOB=flask_example
> + TEST_SCHEDULER_IP=192.168.33.6
> + TEST_ARGS=($TEST_CLUSTER $TEST_ROLE $TEST_ENV $TEST_JOB $TEST_DIR $TEST_SCHEDULER_IP
$EXAMPLE_DIR/flask_example.aurora $EXAMPLE_DIR/flask_example_updated.aurora)
> + trap collect_result EXIT
> + vagrant up
> Bringing machine 'devtools' up with 'virtualbox' provider...
> Bringing machine 'zookeeper' up with 'virtualbox' provider...
> Bringing machine 'mesos-master' up with 'virtualbox' provider...
> Bringing machine 'mesos-slave1' up with 'virtualbox' provider...
> Bringing machine 'mesos-slave2' up with 'virtualbox' provider...
> Bringing machine 'aurora-scheduler' up with 'virtualbox' provider...
> [devtools] VirtualBox VM is already running.
> [zookeeper] VirtualBox VM is already running.
> [mesos-master] VirtualBox VM is already running.
> [mesos-slave1] VirtualBox VM is already running.
> [mesos-slave2] VirtualBox VM is already running.
> [aurora-scheduler] VirtualBox VM is already running.
> + rm -rf deploy_test
> + devtools_setup deploy_test
> + local _testdir=deploy_test
> ++ git branch
> ++ grep '*'
> ++ cut -c 3-
> + branch=v2_e2e
> + run_dev 'if [ ! -d ~/test_dev ]; then git clone /vagrant ~/test_dev; fi'
> + vagrant ssh devtools -c 'if [ ! -d ~/test_dev ]; then git clone /vagrant ~/test_dev;
fi'
> + run_dev 'cd ~/test_dev; git reset --hard; git clean -fdx'
> + vagrant ssh devtools -c 'cd ~/test_dev; git reset --hard; git clean -fdx'
> HEAD is now at 29ae4e5 end-to-end for clientv2
> Removing .BUILD.CPython-2.7.3.pyc
> Removing .pants.d/
> Removing 3rdparty/python/.BUILD.CPython-2.7.3.pyc
> Removing build-support/pants-0.0.14-dev2.pex
> Removing build-support/pex.venv/
> Removing build-support/virtualenv-1.10.1.tar.gz
> Removing build-support/virtualenv-1.10.1/
> Removing dist/
> Removing src/main/python/apache/aurora/client/.BUILD.CPython-2.7.3.pyc
> Removing src/main/python/apache/aurora/client/api/.BUILD.CPython-2.7.3.pyc
> Removing src/main/python/apache/aurora/client/bin/.BUILD.CPython-2.7.3.pyc
> Removing src/main/python/apache/aurora/client/cli/.BUILD.CPython-2.7.3.pyc
> Removing src/main/python/apache/aurora/client/commands/.BUILD.CPython-2.7.3.pyc
> Removing src/main/python/apache/aurora/client/hooks/.BUILD.CPython-2.7.3.pyc
> Removing src/main/python/apache/aurora/common/.BUILD.CPython-2.7.3.pyc
> Removing src/main/python/apache/aurora/common/auth/.BUILD.CPython-2.7.3.pyc
> Removing src/main/python/apache/aurora/config/.BUILD.CPython-2.7.3.pyc
> Removing src/main/python/apache/aurora/config/schema/.BUILD.CPython-2.7.3.pyc
> Removing src/main/python/apache/thermos/common/.BUILD.CPython-2.7.3.pyc
> Removing src/main/python/apache/thermos/config/.BUILD.CPython-2.7.3.pyc
> Removing src/main/thrift/org/apache/aurora/gen/.BUILD.CPython-2.7.3.pyc
> Removing src/main/thrift/org/apache/thermos/.BUILD.CPython-2.7.3.pyc
> Removing src/test/sh/org/apache/aurora/e2e/flask/.BUILD.CPython-2.7.3.pyc
> + run_dev 'cd ~/test_dev ; git checkout v2_e2e; git pull'
> + vagrant ssh devtools -c 'cd ~/test_dev ; git checkout v2_e2e; git pull'
> Already on 'v2_e2e'
> From /vagrant
>    29ae4e5..d9cf866  v2_e2e     -> origin/v2_e2e
> Updating 29ae4e5..d9cf866
> Fast-forward
>  .../sh/org/apache/aurora/e2e/test_end_to_end_v2.sh |  115 ++++++++++++++++++++
>  1 file changed, 115 insertions(+)
>  create mode 100755 src/test/sh/org/apache/aurora/e2e/test_end_to_end_v2.sh
> + run_dev 'cd ~/test_dev; ./pants src/main/python/apache/aurora/client/cli:aurorav2'
> + vagrant ssh devtools -c 'cd ~/test_dev; ./pants src/main/python/apache/aurora/client/cli:aurorav2'
> Building pants.pex @ 0.0.14-dev2
> Bootstrapping twitter.common.python @ 0.3.1
> + VIRTUALENV_VERSION=1.10.1
> + which python2.7
> ++ which python2.7
> + PY=/usr/bin/python2.7
> + echo 'Using /usr/bin/python2.7'
> Using /usr/bin/python2.7
> +++ dirname /home/vagrant/test_dev/build-support/virtualenv
> ++ cd /home/vagrant/test_dev/build-support
> ++ pwd
> + HERE=/home/vagrant/test_dev/build-support
> ~/test_dev/build-support ~/test_dev
> + '[' -f /home/vagrant/test_dev/build-support/virtualenv-1.10.1/BOOTSTRAPPED ']'
> + pushd /home/vagrant/test_dev/build-support
> + curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.10.1.tar.gz
>   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
>                                  Dload  Upload   Total   Spent    Left  Speed
> 100 1294k  100 1294k    0     0  3298k      0 --:--:-- --:--:-- --:--:-- 4215k
> + tar zxvf virtualenv-1.10.1.tar.gz
> virtualenv-1.10.1/
> virtualenv-1.10.1/AUTHORS.txt
> virtualenv-1.10.1/bin/
> virtualenv-1.10.1/bin/rebuild-script.py
> virtualenv-1.10.1/docs/
> virtualenv-1.10.1/docs/_theme/
> virtualenv-1.10.1/docs/_theme/nature/
> virtualenv-1.10.1/docs/_theme/nature/static/
> virtualenv-1.10.1/docs/_theme/nature/static/nature.css_t
> virtualenv-1.10.1/docs/_theme/nature/static/pygments.css
> virtualenv-1.10.1/docs/_theme/nature/theme.conf
> virtualenv-1.10.1/docs/conf.py
> virtualenv-1.10.1/docs/index.rst
> virtualenv-1.10.1/docs/make.bat
> virtualenv-1.10.1/docs/Makefile
> virtualenv-1.10.1/docs/news.rst
> virtualenv-1.10.1/LICENSE.txt
> virtualenv-1.10.1/MANIFEST.in
> virtualenv-1.10.1/PKG-INFO
> virtualenv-1.10.1/README.rst
> virtualenv-1.10.1/scripts/
> virtualenv-1.10.1/scripts/virtualenv
> virtualenv-1.10.1/setup.cfg
> virtualenv-1.10.1/setup.py
> virtualenv-1.10.1/virtualenv.egg-info/
> virtualenv-1.10.1/virtualenv.egg-info/dependency_links.txt
> virtualenv-1.10.1/virtualenv.egg-info/entry_points.txt
> virtualenv-1.10.1/virtualenv.egg-info/not-zip-safe
> virtualenv-1.10.1/virtualenv.egg-info/PKG-INFO
> virtualenv-1.10.1/virtualenv.egg-info/SOURCES.txt
> virtualenv-1.10.1/virtualenv.egg-info/top_level.txt
> virtualenv-1.10.1/virtualenv.py
> virtualenv-1.10.1/virtualenv_embedded/
> virtualenv-1.10.1/virtualenv_embedded/activate.bat
> virtualenv-1.10.1/virtualenv_embedded/activate.csh
> virtualenv-1.10.1/virtualenv_embedded/activate.fish
> virtualenv-1.10.1/virtualenv_embedded/activate.ps1
> virtualenv-1.10.1/virtualenv_embedded/activate.sh
> virtualenv-1.10.1/virtualenv_embedded/activate_this.py
> virtualenv-1.10.1/virtualenv_embedded/deactivate.bat
> virtualenv-1.10.1/virtualenv_embedded/distutils-init.py
> virtualenv-1.10.1/virtualenv_embedded/distutils.cfg
> virtualenv-1.10.1/virtualenv_embedded/site.py
> virtualenv-1.10.1/virtualenv_support/
> virtualenv-1.10.1/virtualenv_support/__init__.py
> virtualenv-1.10.1/virtualenv_support/pip-1.4.1.tar.gz
> virtualenv-1.10.1/virtualenv_support/setuptools-0.9.8.tar.gz
> + touch virtualenv-1.10.1/BOOTSTRAPPED
> ~/test_dev
> + popd
> + exec /usr/bin/python2.7 /home/vagrant/test_dev/build-support/virtualenv-1.10.1/virtualenv.py
/home/vagrant/test_dev/build-support/pex.venv
> New python executable in /home/vagrant/test_dev/build-support/pex.venv/bin/python2.7
> Also creating executable in /home/vagrant/test_dev/build-support/pex.venv/bin/python
> Installing Setuptools..............................................................................................................................................................................................................................done.
> Installing Pip.....................................................................................................................................................................................................................................................................................................................................done.
> Downloading/unpacking twitter.common.python==0.3.1
>   Downloading twitter.common.python-0.3.1.tar.gz
>   Running setup.py egg_info for package twitter.common.python
> 
> Downloading/unpacking setuptools==1.1.7 (from twitter.common.python==0.3.1)
>   Running setup.py egg_info for package setuptools
> 
> Installing collected packages: twitter.common.python, setuptools
>   Running setup.py install for twitter.common.python
>     Skipping installation of /home/vagrant/test_dev/build-support/pex.venv/lib/python2.7/site-packages/twitter/__init__.py
(namespace package)
>     Skipping installation of /home/vagrant/test_dev/build-support/pex.venv/lib/python2.7/site-packages/twitter/common/__init__.py
(namespace package)
> 
>     Installing /home/vagrant/test_dev/build-support/pex.venv/lib/python2.7/site-packages/twitter.common.python-0.3.1-py2.7-nspkg.pth
>     Installing pex script to /home/vagrant/test_dev/build-support/pex.venv/bin
>   Found existing installation: setuptools 0.9.8
>     Uninstalling setuptools:
>       Successfully uninstalled setuptools
>   Running setup.py install for setuptools
> 
>     Installing easy_install script to /home/vagrant/test_dev/build-support/pex.venv/bin
>     Installing easy_install-2.7 script to /home/vagrant/test_dev/build-support/pex.venv/bin
> Successfully installed twitter.common.python setuptools
> Cleaning up...
> Built pants-0.0.14-dev2.pex
> Build operating on targets: OrderedSet([PythonBinary(src/main/python/apache/aurora/client/cli/BUILD:aurorav2)])
> Building PythonBinary PythonBinary(src/main/python/apache/aurora/client/cli/BUILD:aurorav2):
> Wrote /home/vagrant/test_dev/dist/aurorav2.pex
> + run_dev 'cd ~/test_dev; ./pants src/test/sh/org/apache/aurora/e2e/flask:flask_example'
> + vagrant ssh devtools -c 'cd ~/test_dev; ./pants src/test/sh/org/apache/aurora/e2e/flask:flask_example'
> Build operating on targets: OrderedSet([PythonBinary(src/test/sh/org/apache/aurora/e2e/flask/BUILD:flask_example)])
> Building PythonBinary PythonBinary(src/test/sh/org/apache/aurora/e2e/flask/BUILD:flask_example):
> Wrote /home/vagrant/test_dev/dist/flask_example.pex
> + '[' '!' -d deploy_test ']'
> + mkdir deploy_test
> + run_dev 'cd ~/test_dev; cp dist/flask_example.pex /vagrant/deploy_test'
> + vagrant ssh devtools -c 'cd ~/test_dev; cp dist/flask_example.pex /vagrant/deploy_test'
> + run_dev 'cd ~/test_dev; cp dist/aurorav2.pex /vagrant/deploy_test'
> + vagrant ssh devtools -c 'cd ~/test_dev; cp dist/aurorav2.pex /vagrant/deploy_test'
> + test_flask_example example vagrant test flask_example deploy_test 192.168.33.6 /vagrant/src/test/sh/org/apache/aurora/e2e/flask/flask_example.aurora
/vagrant/src/test/sh/org/apache/aurora/e2e/flask/flask_example_updated.aurora
> + local _cluster=example _role=vagrant _env=test _job=flask_example _testdir=deploy_test
_sched_ip=192.168.33.6
> + local _base_config=/vagrant/src/test/sh/org/apache/aurora/e2e/flask/flask_example.aurora
_updated_config=/vagrant/src/test/sh/org/apache/aurora/e2e/flask/flask_example_updated.aurora
> + jobkey=example/vagrant/test/flask_example
> + echo '== Creating job'
> == Creating job
> + run_sched '/vagrant/deploy_test/aurorav2.pex job create example/vagrant/test/flask_example
/vagrant/src/test/sh/org/apache/aurora/e2e/flask/flask_example.aurora'
> + vagrant ssh aurora-scheduler -c '/vagrant/deploy_test/aurorav2.pex job create example/vagrant/test/flask_example
/vagrant/src/test/sh/org/apache/aurora/e2e/flask/flask_example.aurora'
> + base_url=http://192.168.33.6:8081
> ++ _curl -s http://192.168.33.6:8081/scheduler
> ++ wc -l
> ++ curl --silent --fail --retry 4 --retry-delay 10 -s http://192.168.33.6:8081/scheduler
> + schedlen='      50'
> + test 50 -ge 40
> ++ _curl -s http://192.168.33.6:8081/scheduler/vagrant
> ++ wc -l
> ++ curl --silent --fail --retry 4 --retry-delay 10 -s http://192.168.33.6:8081/scheduler/vagrant
> + rolelen='     195'
> + test 195 -ge 50
> ++ _curl http://192.168.33.6:8081/scheduler/vagrant/test/flask_example
> ++ curl --silent --fail --retry 4 --retry-delay 10 http://192.168.33.6:8081/scheduler/vagrant/test/flask_example
> ++ wc -l
> + joblen='     526'
> + test 526 -ge 100
> + echo '== Updating test job'
> == Updating test job
> + run_sched '/vagrant/deploy_test/aurorav2.pex job update example/vagrant/test/flask_example
/vagrant/src/test/sh/org/apache/aurora/e2e/flask/flask_example_updated.aurora'
> + vagrant ssh aurora-scheduler -c '/vagrant/deploy_test/aurorav2.pex job update example/vagrant/test/flask_example
/vagrant/src/test/sh/org/apache/aurora/e2e/flask/flask_example_updated.aurora'
> Warning: this update is a large change. Press ^C within 5 seconds to abort
> ++ wc -l
> ++ run_sched '/vagrant/deploy_test/aurorav2.pex task run example/vagrant/test/flask_example
'\''pwd'\'''
> ++ vagrant ssh aurora-scheduler -c '/vagrant/deploy_test/aurorav2.pex task run example/vagrant/test/flask_example
'\''pwd'\'''
> + runlen='       2'
> + test 2 -eq 2
> + run_sched '/vagrant/deploy_test/aurorav2.pex quota get example/vagrant'
> + vagrant ssh aurora-scheduler -c '/vagrant/deploy_test/aurorav2.pex quota get example/vagrant'
> Allocated:
>   CPU: 0.0
>   RAM: 0.000000 GB
>   Disk: 0.000000 GB
> Production resources consumed:
>   CPU: 0.0
>   RAM: 0.000000 GB
>   Disk: 0.000000 GB
> Non-production resources consumed:
>   CPU: 2.0
>   RAM: 0.125000 GB
>   Disk: 0.187500 GB
> + run_sched '/vagrant/deploy_test/aurorav2.pex job killall  example/vagrant/test/flask_example'
> + vagrant ssh aurora-scheduler -c '/vagrant/deploy_test/aurorav2.pex job killall  example/vagrant/test/flask_example'
> + RETCODE=0
> + collect_result
> + [[ 0 = 0 ]]
> + echo '***'
> ***
> + echo 'OK (all tests passed)'
> OK (all tests passed)
> + echo '***'
> ***
> + exit 0
> 
> 
> Thanks,
> 
> Mark Chu-Carroll
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message