Return-Path: X-Original-To: apmail-incubator-allura-commits-archive@minotaur.apache.org Delivered-To: apmail-incubator-allura-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2E46C102BE for ; Mon, 15 Jul 2013 21:03:59 +0000 (UTC) Received: (qmail 95610 invoked by uid 500); 15 Jul 2013 21:03:59 -0000 Delivered-To: apmail-incubator-allura-commits-archive@incubator.apache.org Received: (qmail 95527 invoked by uid 500); 15 Jul 2013 21:03:59 -0000 Mailing-List: contact allura-commits-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: allura-dev@incubator.apache.org Delivered-To: mailing list allura-commits@incubator.apache.org Received: (qmail 94668 invoked by uid 99); 15 Jul 2013 21:03:58 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 15 Jul 2013 21:03:58 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id 42A758A8252; Mon, 15 Jul 2013 21:03:58 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: tvansteenburgh@apache.org To: allura-commits@incubator.apache.org Date: Mon, 15 Jul 2013 21:04:41 -0000 Message-Id: <4fc5f18b44764119b16367325779da12@git.apache.org> In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [45/50] [abbrv] git commit: [#4122] rename AlluraTesting dir to AlluraTest, to match the module name below [#4122] rename AlluraTesting dir to AlluraTest, to match the module name below This makes the coverage output from ./run_tests better Project: http://git-wip-us.apache.org/repos/asf/incubator-allura/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-allura/commit/37e91c91 Tree: http://git-wip-us.apache.org/repos/asf/incubator-allura/tree/37e91c91 Diff: http://git-wip-us.apache.org/repos/asf/incubator-allura/diff/37e91c91 Branch: refs/heads/tv/6355 Commit: 37e91c9136b973d84eb7309858ddb1e20f05c586 Parents: c2ac016 Author: Dave Brondsema Authored: Mon May 6 18:24:09 2013 -0400 Committer: Dave Brondsema Committed: Thu Jul 11 21:30:45 2013 +0000 ---------------------------------------------------------------------- AlluraTest/LICENSE | 234 + AlluraTest/alluratest/__init__.py | 16 + AlluraTest/alluratest/controller.py | 183 + AlluraTest/alluratest/test_syntax.py | 95 + AlluraTest/alluratest/validation.py | 321 + AlluraTest/jslint/Makefile | 7 + AlluraTest/jslint/env-js.jar | Bin 0 -> 1110999 bytes AlluraTest/jslint/fulljslint.js | 5688 +++++++++++++++++ AlluraTest/jslint/js.jar | Bin 0 -> 871260 bytes AlluraTest/jslint/jslint.js | 5729 ++++++++++++++++++ AlluraTest/jslint/rhino.js | 41 + AlluraTest/setup.py | 39 + AlluraTest/twill-tests/README | 19 + AlluraTest/twill-tests/create_repo.twill | 47 + AlluraTest/twill-tests/edit_wiki_page.twill | 36 + AlluraTest/twill-tests/login.twill | 27 + AlluraTest/twill-tests/new_issue.twill | 36 + AlluraTest/twill-tests/smoke-front-page.twill | 20 + AlluraTest/twill-tests/smoke-project-home.twill | 20 + .../twill-tests/smoke-tracker-search.twill | 20 + AlluraTest/twill-tests/smoke-tracker.twill | 20 + AlluraTest/twill-tests/smoke-user-profile.twill | 20 + AlluraTest/twill-tests/smoke-wiki.twill | 20 + AlluraTesting/LICENSE | 234 - AlluraTesting/alluratest/__init__.py | 16 - AlluraTesting/alluratest/controller.py | 183 - AlluraTesting/alluratest/test_syntax.py | 95 - AlluraTesting/alluratest/validation.py | 321 - AlluraTesting/jslint/Makefile | 7 - AlluraTesting/jslint/env-js.jar | Bin 1110999 -> 0 bytes AlluraTesting/jslint/fulljslint.js | 5688 ----------------- AlluraTesting/jslint/js.jar | Bin 871260 -> 0 bytes AlluraTesting/jslint/jslint.js | 5729 ------------------ AlluraTesting/jslint/rhino.js | 41 - AlluraTesting/setup.py | 39 - AlluraTesting/twill-tests/README | 19 - AlluraTesting/twill-tests/create_repo.twill | 47 - AlluraTesting/twill-tests/edit_wiki_page.twill | 36 - AlluraTesting/twill-tests/login.twill | 27 - AlluraTesting/twill-tests/new_issue.twill | 36 - .../twill-tests/smoke-front-page.twill | 20 - .../twill-tests/smoke-project-home.twill | 20 - .../twill-tests/smoke-tracker-search.twill | 20 - AlluraTesting/twill-tests/smoke-tracker.twill | 20 - .../twill-tests/smoke-user-profile.twill | 20 - AlluraTesting/twill-tests/smoke-wiki.twill | 20 - rat-excludes.txt | 2 +- run_clonedigger | 2 +- run_tests | 2 +- 49 files changed, 12641 insertions(+), 12641 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-allura/blob/37e91c91/AlluraTest/LICENSE ---------------------------------------------------------------------- diff --git a/AlluraTest/LICENSE b/AlluraTest/LICENSE new file mode 100644 index 0000000..15f71ee --- /dev/null +++ b/AlluraTest/LICENSE @@ -0,0 +1,234 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + +AlluraTest SUBCOMPONENTS: + +AlluraTest includes a number of subcomponents with +separate copyright notices and license terms. Your use of the source code +for the these subcomponents is subject to the terms and conditions of the +following licenses. + +For jslint.js, in directory +jslint/ + + Copyright (c) 2002 Douglas Crockford (www.JSLint.com) + + Permission is hereby granted, free of charge, to any person obtaining a copy of + this software and associated documentation files (the "Software"), to deal in + the Software without restriction, including without limitation the rights to + use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies + of the Software, and to permit persons to whom the Software is furnished to do + so, subject to the following conditions: + + The above copyright notice and this permission notice shall be included in all + copies or substantial portions of the Software. + + The Software shall be used for Good, not Evil. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + SOFTWARE. http://git-wip-us.apache.org/repos/asf/incubator-allura/blob/37e91c91/AlluraTest/alluratest/__init__.py ---------------------------------------------------------------------- diff --git a/AlluraTest/alluratest/__init__.py b/AlluraTest/alluratest/__init__.py new file mode 100644 index 0000000..144e298 --- /dev/null +++ b/AlluraTest/alluratest/__init__.py @@ -0,0 +1,16 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. http://git-wip-us.apache.org/repos/asf/incubator-allura/blob/37e91c91/AlluraTest/alluratest/controller.py ---------------------------------------------------------------------- diff --git a/AlluraTest/alluratest/controller.py b/AlluraTest/alluratest/controller.py new file mode 100644 index 0000000..95d9b4b --- /dev/null +++ b/AlluraTest/alluratest/controller.py @@ -0,0 +1,183 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +"""Unit and functional test suite for allura.""" +import os +import urllib + +import mock +import beaker.session +from formencode import variabledecode +from paste.deploy import loadapp +from paste.deploy.converters import asbool +from paste.script.appinstall import SetupCommand +from pylons import tmpl_context as c, app_globals as g +from pylons import url, request, response, session +import tg +from webtest import TestApp +from webob import Request, Response +import ew +from ming.orm import ThreadLocalORMSession +import ming.orm + +from allura import model as M +import allura.lib.security +from allura.lib.app_globals import Globals +from allura.lib import helpers as h +from allura.websetup.schema import REGISTRY +#from allura.lib.custom_middleware import environ as ENV, MagicalC +from .validation import ValidatingTestApp + +DFL_APP_NAME = 'main_without_authn' + +# these are all helpers & base classes, and should never +# be considered test cases when imported into some test module +__test__ = False + + +def get_config_file(config=None): + if not config: + config = 'test.ini' + + try: + conf_dir = tg.config.here + except AttributeError: + conf_dir = os.getcwd() + return os.path.join(conf_dir, config) + + +def setup_basic_test(config=None, app_name=DFL_APP_NAME): + '''Create clean environment for running tests''' + try: + conf_dir = tg.config.here + except AttributeError: + conf_dir = os.getcwd() + ew.TemplateEngine.initialize({}) + test_file = os.path.join(conf_dir, get_config_file(config)) + cmd = SetupCommand('setup-app') + cmd.run([test_file]) + + # run all tasks, e.g. indexing from bootstrap operations + while M.MonQTask.run_ready('setup'): + ThreadLocalORMSession.flush_all() +setup_basic_test.__test__ = False # sometimes __test__ above isn't sufficient + + +def setup_functional_test(config=None, app_name=DFL_APP_NAME): + '''Create clean environment for running tests. Also return WSGI test app''' + config = get_config_file(config) + setup_basic_test(config, app_name) + conf_dir = tg.config.here + wsgiapp = loadapp('config:%s#%s' % (config, app_name), + relative_to=conf_dir) + return wsgiapp +setup_functional_test.__test__ = False # sometimes __test__ above isn't sufficient + + +def setup_unit_test(): + try: + while True: + REGISTRY.cleanup() + except: + pass + REGISTRY.prepare() + REGISTRY.register(ew.widget_context, ew.core.WidgetContext('http', ew.ResourceManager())) + REGISTRY.register(g, Globals()) + REGISTRY.register(c, mock.Mock()) + REGISTRY.register(url, lambda:None) + REGISTRY.register(request, Request.blank('/', remote_addr='1.1.1.1')) + REGISTRY.register(response, Response()) + REGISTRY.register(session, beaker.session.SessionObject({})) + REGISTRY.register(allura.credentials, allura.lib.security.Credentials()) + c.memoize_cache = {} + c.queued_messages = None + c.model_cache = None + ThreadLocalORMSession.close_all() +setup_unit_test.__test__ = False # sometimes __test__ above isn't sufficient + + +def setup_global_objects(): + setup_unit_test() + h.set_context('test', 'wiki', neighborhood='Projects') + c.user = M.User.query.get(username='test-admin') + + +class TestController(object): + + application_under_test = 'main' + validate_skip = False + + def setUp(self): + """Method called by nose before running each test""" + self.app = ValidatingTestApp(setup_functional_test(app_name=self.application_under_test)) + if self.validate_skip: + self.app.validate_skip = self.validate_skip + if asbool(tg.config.get('smtp.mock')): + self.smtp_mock = mock.patch('allura.lib.mail_util.smtplib.SMTP') + self.smtp_mock.start() + + def tearDown(self): + """Method called by nose after running each test""" + if asbool(tg.config.get('smtp.mock')): + self.smtp_mock.stop() + + def webflash(self, response): + "Extract webflash content from response." + return urllib.unquote(response.cookies_set.get('webflash', '')) + + +class TestRestApiBase(TestController): + + def setUp(self): + super(TestRestApiBase, self).setUp() + setup_global_objects() +# h.set_context('test', 'home') + self.user = M.User.query.get(username='test-admin') + self.token = M.ApiToken(user_id=self.user._id) + ming.orm.session(self.token).flush() + + def set_api_token(self, token): + self.token = token + + def _api_getpost(self, method, path, api_key=None, api_timestamp=None, api_signature=None, + wrap_args=None, **params): + if wrap_args: + params = {wrap_args: params} + params = variabledecode.variable_encode(params, add_repetitions=False) + if api_key: params['api_key'] = api_key + if api_timestamp: params['api_timestamp'] = api_timestamp + if api_signature: params['api_signature'] = api_signature + params = self.token.sign_request(path, params) + + fn = self.app.post if method=='POST' else self.app.get + + response = fn( + str(path), + params=params, + status=[200, 302, 400, 403, 404]) + if response.status_int == 302: + return response.follow() + else: + return response + + def api_get(self, path, api_key=None, api_timestamp=None, api_signature=None, + wrap_args=None, **params): + return self._api_getpost('GET', path, api_key, api_timestamp, api_signature, wrap_args, **params) + + def api_post(self, path, api_key=None, api_timestamp=None, api_signature=None, + wrap_args=None, **params): + return self._api_getpost('POST', path, api_key, api_timestamp, api_signature, wrap_args, **params) http://git-wip-us.apache.org/repos/asf/incubator-allura/blob/37e91c91/AlluraTest/alluratest/test_syntax.py ---------------------------------------------------------------------- diff --git a/AlluraTest/alluratest/test_syntax.py b/AlluraTest/alluratest/test_syntax.py new file mode 100644 index 0000000..b394798 --- /dev/null +++ b/AlluraTest/alluratest/test_syntax.py @@ -0,0 +1,95 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +import os.path +from glob import glob +from subprocess import Popen, PIPE +import sys + +toplevel_dir = os.path.abspath(os.path.dirname(__file__) + "/../..") + +def run(cmd): + proc = Popen(cmd, shell=True, cwd=toplevel_dir, stdout=PIPE, stderr=PIPE) + # must capture & reprint stdount, so that nosetests can capture it + (stdout, stderr) = proc.communicate() + sys.stdout.write(stdout) + sys.stderr.write(stderr) + return proc.returncode + +find_py = "find Allura Forge* -name '*.py'" + +# a recepe from itertools doc +from itertools import izip_longest +def grouper(n, iterable, fillvalue=None): + "grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx" + args = [iter(iterable)] * n + return izip_longest(fillvalue=fillvalue, *args) + +def test_pyflakes(): + # skip some that aren't critical errors + skips = [ + 'imported but unused', + 'redefinition of unused', + 'assigned to but never used', + '__version__', + ] + proc = Popen(find_py, shell=True, cwd=toplevel_dir, stdout=PIPE, stderr=PIPE) + (find_stdout, stderr) = proc.communicate() + sys.stderr.write(stderr) + assert proc.returncode == 0, proc.returncode + + # run pyflakes in batches, so it doesn't take tons of memory + error = False + all_files = [f for f in find_stdout.split('\n') + if '/migrations/' not in f and f.strip()] + for files in grouper(20, all_files, fillvalue=''): + cmd = "pyflakes " + ' '.join(files) + " | grep -v '" + "' | grep -v '".join(skips) + "'" + #print 'Command was: %s' % cmd + retval = run(cmd) + if retval != 1: + print + #print 'Command was: %s' % cmd + print 'Returned %s' % retval + error = True + + if error: + raise Exception('pyflakes failure, see stdout') + +def test_no_now(): + if run(find_py + " | xargs grep '\.now(' ") not in [1,123]: + raise Exception("These should use .utcnow()") + if run(find_py + " | xargs grep '\.fromtimestamp(' ") not in [1,123]: + raise Exception("These should use .utcfromtimestamp()") + +def test_no_prints(): + skips = [ + '/tests/', + 'Allura/allura/command/', + 'Allura/ldap-setup.py', + 'Allura/ldap-userconfig.py', + 'Allura/ez_setup/', + 'Allura/allura/lib/AsciiDammit.py', + '/scripts/', + 'Allura/allura/lib/import_api.py', + 'ForgeSVN/setup.py', + ] + if run(find_py + " | grep -v '" + "' | grep -v '".join(skips) + "' | xargs grep -v '^ *#' | grep 'print ' | grep -E -v '(pprint|#pragma: ?printok)' ") != 1: + raise Exception("These should use logging instead of print") + +def test_no_tabs(): + if run(find_py + " | xargs grep ' ' ") not in [1,123]: + raise Exception('These should not use tab chars') http://git-wip-us.apache.org/repos/asf/incubator-allura/blob/37e91c91/AlluraTest/alluratest/validation.py ---------------------------------------------------------------------- diff --git a/AlluraTest/alluratest/validation.py b/AlluraTest/alluratest/validation.py new file mode 100644 index 0000000..235d7c0 --- /dev/null +++ b/AlluraTest/alluratest/validation.py @@ -0,0 +1,321 @@ +# -*- coding: utf-8 -*- + +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +""" +Functions to syntax-validate output content +""" +from os import path, environ, getcwd +import os +import sys +import logging +import tempfile +import subprocess +import json +import urllib2 +import re + +import tg +import mock +import beaker.session +from paste.deploy import loadapp +from paste.script.appinstall import SetupCommand +from pylons import tmpl_context as c, app_globals as g +from pylons import url, request, response, session +import webtest +from webtest import TestApp +from webob import Request, Response +from nose.tools import ok_, assert_true, assert_false +from poster.encode import multipart_encode +from poster.streaminghttp import register_openers +from ming.utils import LazyProperty + +from allura.lib import utils + +ENABLE_CONTENT_VALIDATION = False +# By default we want to run only validations which are fast, +# but on special test hosts - all. +COMPLETE_TESTS_HOST = 'sb-forge-4039' + +log = logging.getLogger(__name__) + +class Config(object): + "Config to encapsulate flexible/complex test enabled/disabled rules." + _instance = None + + def __init__(self): + self.ini_config = None + pass + + @classmethod + def instance(cls): + if not cls._instance: + cls._instance = cls() + return cls._instance + + @LazyProperty + def test_ini(self): + if not self.ini_config: + from . import controller + import ConfigParser + conf = ConfigParser.ConfigParser({'validate_html5': 'false', 'validate_inlinejs': 'false'}) + conf.read(controller.get_config_file()) + self.ini_config = conf + return self.ini_config + + @LazyProperty + def hostname(self): + if os.path.exists('/etc/soghost'): + with open('/etc/soghost') as fp: + return fp.read().strip() + + def validation_enabled(self, val_type): + env_var = os.getenv('ALLURA_VALIDATION') + if env_var == 'all': + return True + elif env_var == 'none': + return False + elif env_var is not None: + return val_type in env_var.split(',') + + if self.hostname == COMPLETE_TESTS_HOST: + return True + + enabled = self.test_ini.getboolean('validation', 'validate_' + val_type) + return enabled + + def fail_on_validation(self, val_type): + env_var = os.getenv('ALLURA_VALIDATION') + if env_var == 'all': + return True + if self.hostname == COMPLETE_TESTS_HOST: + return True + return ENABLE_CONTENT_VALIDATION + + +def report_validation_error(val_name, filename, message): + message = '%s Validation errors (%s):\n%s\n' % (val_name, filename, message) + if Config.instance().fail_on_validation(val_name): + ok_(False, message) + else: + sys.stderr.write('=' * 40 + '\n' + message) + +def dump_to_file(prefix, html): + f = tempfile.NamedTemporaryFile(prefix=prefix, delete=False) + f.write(html) + f.close() + return f.name + +def validate_html(html_or_response): + if hasattr(html_or_response, 'body'): + html = html_or_response.body + else: + html = html_or_response + + html = html.lstrip() + + if html.startswith(''): + return validate_html5(html) + else: + assert False, 'Non-valid HTML: ' + html[:100] + '...' + +def validate_json(json_or_response): + if hasattr(json_or_response, 'body'): + j = json_or_response.body + else: + j = json_or_response + + try: + obj = json.loads(j) + except Exception, e: + ok_(False, "Couldn't validate JSON: " + str(e) + ':' + j[:100] + '...') + + return obj + +def validate_html5(html_or_response): + if hasattr(html_or_response, 'body'): + html = html_or_response.body + else: + html = html_or_response + register_openers() + params = [("out","text"),("content",html)] + datagen, headers = multipart_encode(params) + request = urllib2.Request("http://html5.validator.nu/", datagen, headers) + count = 3 + while True: + try: + resp = urllib2.urlopen(request, timeout=3).read() + break + except: + resp = "Couldn't connect to validation service to check the HTML" + count -= 1 + if count == 0: + sys.stderr.write('WARNING: ' + resp + '\n') + break + + resp = resp.replace('“','"').replace('”','"').replace('–','-') + + ignored_errors = [ + 'Required attributes missing on element "object"', + 'Stray end tag "embed".', + 'Stray end tag "param".', + r'Bad value .+? for attribute "onclick" on element "input": invalid return', + ] + for ignore in ignored_errors: + resp = re.sub('Error: ' + ignore, 'Ignoring: ' + ignore, resp) + + if 'Error:' in resp: + fname = dump_to_file('html5-', html) + message = resp.decode('ascii','ignore') + report_validation_error('html5', fname, message) + + +def validate_html5_chunk(html): + """ When you don't have a html & body tags - this adds it""" + # WebTest doesn't like HTML fragments without doctype, + # so we output them sometimes for fragments, which is hack. + # Unhack it here. + doctype = '' + if html.startswith(doctype): + html = html[len(doctype):] + + html = ''' + + + + %s + ''' % html + return validate_html5(html) + +def validate_js(html_or_response): + if hasattr(html_or_response, 'body'): + if html_or_response.status_int != 200: + return + html = html_or_response.body + else: + html = html_or_response + basedir = path.dirname(path.abspath(__file__)) + jslint_dir = basedir + '/../jslint' + fname = dump_to_file('jslint-', html) + cmd = 'java -jar ' + jslint_dir + '/js.jar '+ jslint_dir +'/jslint.js ' + fname + p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + stdout, stderr = p.communicate(html) + if stdout.startswith('jslint: No problems found'): + os.unlink(fname) + return + stdout = stdout.decode('UTF-8', 'replace') + msg = '\n'.join(repr(s) for s in stdout.split('\n') if s) + report_validation_error('js', fname, msg) + +def validate_page(html_or_response): + if Config.instance().validation_enabled('html5'): + validate_html(html_or_response) + if Config.instance().validation_enabled('inlinejs'): + validate_js(html_or_response) + +class AntiSpamTestApp(TestApp): + + def post(self, *args, **kwargs): + if kwargs.pop('antispam', False): + antispam = utils.AntiSpam() + params = { + 'timestamp': antispam.timestamp_text, + 'spinner': antispam.spinner_text, + antispam.enc('honey0'): '', + antispam.enc('honey1'): '', + } + for k, v in kwargs['params'].iteritems(): + params[antispam.enc(k)] = v + kwargs['params'] = params + return super(AntiSpamTestApp, self).post(*args, **kwargs) + +class PostParamCheckingTestApp(AntiSpamTestApp): + + def _validate_params(self, params, method): + if not params: + return + # params can be a list or a dict + if hasattr(params, 'items'): + params = params.items() + for k, v in params: + if not isinstance(k, basestring): + raise TypeError('%s key %s is %s, not str' % (method, k, type(k))) + if not isinstance(v, (basestring, webtest.app.File)): + raise TypeError('%s key %s has value %s of type %s, not str. ' % (method, k, v, type(v))) + + def get(self, *args, **kwargs): + self._validate_params(kwargs.get('params'), 'get') + return super(PostParamCheckingTestApp, self).get(*args, **kwargs) + + def post(self, *args, **kwargs): + self._validate_params(kwargs.get('params'), 'post') + return super(PostParamCheckingTestApp, self).post(*args, **kwargs) + +class ValidatingTestApp(PostParamCheckingTestApp): + + # Subclasses may set this to True to skip validation altogether + validate_skip = False + + def _validate(self, resp, method, val_params): + """Perform validation on webapp response. This handles responses of + various types and forms.""" + if resp.status_int != 200: + return + + content = resp.body + content_type = resp.headers['Content-Type'] + if content_type.startswith('text/html'): + if val_params['validate_chunk']: + validate_html5_chunk(content) + else: + validate_page(resp) + elif content_type.split(';', 1)[0] in ('text/plain', 'text/x-python', 'application/octet-stream'): + pass + elif content_type.startswith('application/json'): + validate_json(content) + elif content_type.startswith(('application/x-javascript','application/javascript', 'text/javascript')): + validate_js(content) + elif content_type.startswith('application/xml'): + import feedparser + d = feedparser.parse(content) + assert d.bozo == 0, 'Non-wellformed feed' + elif content_type.startswith('image/'): + pass + else: + assert False, 'Unexpected output content type: ' + content_type + + def _get_validation_params(self, kw): + "Separate validation params from normal TestApp methods params." + params = {} + for k in ('validate_skip', 'validate_chunk'): + params[k] = kw.pop(k, False) + return params, kw + + def get(self, *args, **kw): + val_params, kw = self._get_validation_params(kw) + resp = super(ValidatingTestApp, self).get(*args, **kw) + if not self.validate_skip and not val_params['validate_skip']: + self._validate(resp, 'get', val_params) + return resp + + def post(self, *args, **kw): + val_params, kw = self._get_validation_params(kw) + resp = super(ValidatingTestApp, self).post(*args, **kw) + if not self.validate_skip and not val_params['validate_skip']: + self._validate(resp, 'post', val_params) + return resp http://git-wip-us.apache.org/repos/asf/incubator-allura/blob/37e91c91/AlluraTest/jslint/Makefile ---------------------------------------------------------------------- diff --git a/AlluraTest/jslint/Makefile b/AlluraTest/jslint/Makefile new file mode 100644 index 0000000..7efd72e --- /dev/null +++ b/AlluraTest/jslint/Makefile @@ -0,0 +1,7 @@ +# Make single-file jslint script out of main script (from jslint.com, +# patched by us to tweak its behavior, as it configurability leaves much +# to be desired) and Rhino support module. Needed because Rhino accepts +# only single file as script. + +jslint.js: fulljslint.js rhino.js + cat $^ >$@ http://git-wip-us.apache.org/repos/asf/incubator-allura/blob/37e91c91/AlluraTest/jslint/env-js.jar ---------------------------------------------------------------------- diff --git a/AlluraTest/jslint/env-js.jar b/AlluraTest/jslint/env-js.jar new file mode 100644 index 0000000..072891a Binary files /dev/null and b/AlluraTest/jslint/env-js.jar differ