Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 46CEC200CC5 for ; Tue, 11 Jul 2017 15:09:02 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 455261610A3; Tue, 11 Jul 2017 13:09:02 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 65052160C38 for ; Tue, 11 Jul 2017 15:09:01 +0200 (CEST) Received: (qmail 67662 invoked by uid 500); 11 Jul 2017 13:08:59 -0000 Mailing-List: contact commits-help@ariatosca.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ariatosca.incubator.apache.org Delivered-To: mailing list commits@ariatosca.incubator.apache.org Received: (qmail 67653 invoked by uid 99); 11 Jul 2017 13:08:59 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Jul 2017 13:08:59 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 8E1801960E3 for ; Tue, 11 Jul 2017 13:08:58 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.221 X-Spam-Level: X-Spam-Status: No, score=-4.221 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id O_utwv-18Rlt for ; Tue, 11 Jul 2017 13:08:54 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id EBABA62481 for ; Tue, 11 Jul 2017 13:01:37 +0000 (UTC) Received: (qmail 32289 invoked by uid 99); 11 Jul 2017 13:01:37 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Jul 2017 13:01:37 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 17316DFC33; Tue, 11 Jul 2017 13:01:37 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: emblemparade@apache.org To: commits@ariatosca.incubator.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: incubator-ariatosca git commit: ARIA-76 Parallelize PyTest [Forced Update!] Date: Tue, 11 Jul 2017 13:01:37 +0000 (UTC) archived-at: Tue, 11 Jul 2017 13:09:02 -0000 Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-76-parallel-pytest 541b2dda8 -> a447ebe80 (forced update) ARIA-76 Parallelize PyTest Use the PyTest xdist plugin to parallelize tests in boxed subprocesses. Through benchmarking we discovered that paralellizing on the number of CPU cores ("-n auto") provides the best times. Also update all our testing dependencies. Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/a447ebe8 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/a447ebe8 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/a447ebe8 Branch: refs/heads/ARIA-76-parallel-pytest Commit: a447ebe80127d7a81f5ff3a3995699b8122c49d6 Parents: 6c2f35e Author: Tal Liron Authored: Mon Jul 10 18:06:05 2017 +0300 Committer: Tal Liron Committed: Tue Jul 11 16:01:25 2017 +0300 ---------------------------------------------------------------------- .travis.yml | 2 ++ tests/requirements.txt | 17 ++++++----- tox.ini | 74 ++++++++++++++++++++++++++++++++++++--------- 3 files changed, 70 insertions(+), 23 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/a447ebe8/.travis.yml ---------------------------------------------------------------------- diff --git a/.travis.yml b/.travis.yml index de02d78..1401790 100644 --- a/.travis.yml +++ b/.travis.yml @@ -23,6 +23,8 @@ env: - TOX_ENV=py26e2e - TOX_ENV=py27ssh - TOX_ENV=py26ssh +- TOX_ENV=docs +- PYTEST_PROCESSES=2 install: - pip install --upgrade pip - pip install --upgrade setuptools http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/a447ebe8/tests/requirements.txt ---------------------------------------------------------------------- diff --git a/tests/requirements.txt b/tests/requirements.txt index cf57821..d86750b 100644 --- a/tests/requirements.txt +++ b/tests/requirements.txt @@ -10,12 +10,13 @@ # See the License for the specific language governing permissions and # limitations under the License. -testtools -fasteners==0.13.0 -sh==1.12.13 +testtools==2.3.0 +fasteners==0.14.1 +sh==1.12.14 psutil==5.2.2 -mock==1.0.1 -pylint==1.6.4 -pytest==3.0.2 -pytest-cov==2.3.1 -pytest-mock==1.2 +mock==2.0.0 +pylint==1.6.5 +pytest==3.1.3 +pytest-cov==2.5.1 +pytest-mock==1.6.0 +pytest-xdist==1.18.1 http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/a447ebe8/tox.ini ---------------------------------------------------------------------- diff --git a/tox.ini b/tox.ini index 9849b5e..129c557 100644 --- a/tox.ini +++ b/tox.ini @@ -12,17 +12,21 @@ [tox] envlist=py27,py26,py27e2e,py26e2e,pywin,py27ssh,pylint_code,pylint_tests,docs +processes={env:PYTEST_PROCESSES:auto} [testenv] -whitelist_externals=rm +whitelist_externals= + rm passenv= TRAVIS PYTHON PYTHON_VERSION PYTHON_ARCH deps= - -rrequirements.txt - -rtests/requirements.txt + --requirement + requirements.txt + --requirement + tests/requirements.txt basepython= py26: python2.6 py27: python2.7 @@ -36,36 +40,76 @@ basepython= docs: python2.7 [testenv:py27] -commands=pytest tests --ignore=tests/end2end --ignore=tests/orchestrator/execution_plugin/test_ssh.py --cov-report term-missing --cov aria +commands= + pytest tests \ + --numprocesses={[tox]processes} \ + --ignore=tests/end2end \ + --ignore=tests/orchestrator/execution_plugin/test_ssh.py \ + --cov-report term-missing \ + --cov aria [testenv:py26] -commands=pytest tests --ignore=tests/end2end --ignore=tests/orchestrator/execution_plugin/test_ssh.py --cov-report term-missing --cov aria +commands= + pytest tests \ + --numprocesses={[tox]processes} \ + --ignore=tests/end2end \ + --ignore=tests/orchestrator/execution_plugin/test_ssh.py \ + --cov-report term-missing \ + --cov aria [testenv:py27e2e] -commands=pytest tests/end2end --cov-report term-missing --cov aria +commands= + pytest tests/end2end \ + --numprocesses={[tox]processes} \ + --cov-report term-missing \ + --cov aria [testenv:py26e2e] -commands=pytest tests/end2end --cov-report term-missing --cov aria +commands= + pytest tests/end2end \ + --numprocesses={[tox]processes} \ + --cov-report term-missing \ + --cov aria [testenv:pywin] -commands=pytest tests --ignore=tests/end2end --ignore=tests/orchestrator/execution_plugin/test_ssh.py --cov-report term-missing --cov aria +commands= + pytest tests \ + --numprocesses={[tox]processes} \ + --ignore=tests/end2end \ + --ignore=tests/orchestrator/execution_plugin/test_ssh.py \ + --cov-report term-missing \ + --cov aria [testenv:py27ssh] -install_command=pip install {opts} {packages} .[ssh] -commands=pytest tests/orchestrator/execution_plugin/test_ssh.py +install_command= + pip install {opts} {packages} .[ssh] +commands= + pytest tests/orchestrator/execution_plugin/test_ssh.py \ + --numprocesses={[tox]processes} [testenv:py26ssh] -install_command=pip install {opts} {packages} .[ssh] -commands=pytest tests/orchestrator/execution_plugin/test_ssh.py +install_command= + pip install {opts} {packages} .[ssh] +commands= + pytest tests/orchestrator/execution_plugin/test_ssh.py \ + --numprocesses={[tox]processes} [testenv:pylint_code] -commands=pylint --rcfile=aria/.pylintrc --disable=fixme,missing-docstring aria extensions/aria_extension_tosca/ +commands= + pylint aria extensions/aria_extension_tosca/ \ + --rcfile=aria/.pylintrc \ + --disable=fixme,missing-docstring [testenv:pylint_tests] -commands=pylint --rcfile=tests/.pylintrc --disable=fixme,missing-docstring tests +commands= + pylint tests \ + --rcfile=tests/.pylintrc \ + --disable=fixme,missing-docstring [testenv:docs] +install_command= + pip install {opts} {packages} \ + --requirement docs/requirements.txt commands= - pip install --requirement docs/requirements.txt rm --recursive --force docs/html sphinx-build -W -T -b html docs docs/html