Return-Path: X-Original-To: apmail-subversion-commits-archive@minotaur.apache.org Delivered-To: apmail-subversion-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id BBE0118930 for ; Mon, 30 Nov 2015 10:25:15 +0000 (UTC) Received: (qmail 56777 invoked by uid 500); 30 Nov 2015 10:25:15 -0000 Delivered-To: apmail-subversion-commits-archive@subversion.apache.org Received: (qmail 56748 invoked by uid 500); 30 Nov 2015 10:25:15 -0000 Mailing-List: contact commits-help@subversion.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@subversion.apache.org Delivered-To: mailing list commits@subversion.apache.org Received: (qmail 56738 invoked by uid 99); 30 Nov 2015 10:25:15 -0000 Received: from Unknown (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 30 Nov 2015 10:25:15 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id D251018098E for ; Mon, 30 Nov 2015 10:25:14 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.497 X-Spam-Level: X-Spam-Status: No, score=0.497 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LOTSOFHASH=0.25, RP_MATCHES_RCVD=-0.554, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id hRgQLEvqWMvn for ; Mon, 30 Nov 2015 10:24:46 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTP id 16C7D25EA7 for ; Mon, 30 Nov 2015 10:24:32 +0000 (UTC) Received: from svn01-us-west.apache.org (svn.apache.org [10.41.0.6]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id D406BE109F for ; Mon, 30 Nov 2015 10:24:30 +0000 (UTC) Received: from svn01-us-west.apache.org (localhost [127.0.0.1]) by svn01-us-west.apache.org (ASF Mail Server at svn01-us-west.apache.org) with ESMTP id CBA5B3A04A4 for ; Mon, 30 Nov 2015 10:24:30 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: svn commit: r1717223 [50/50] - in /subversion/branches/ra-git: ./ build/ build/ac-macros/ build/generator/ build/generator/templates/ contrib/hook-scripts/ notes/ notes/api-errata/1.9/ notes/move-tracking/ subversion/ subversion/bindings/ctypes-python/... Date: Mon, 30 Nov 2015 10:24:23 -0000 To: commits@subversion.apache.org From: rhuijben@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20151130102430.CBA5B3A04A4@svn01-us-west.apache.org> Modified: subversion/branches/ra-git/tools/dev/unix-build/Makefile.svn URL: http://svn.apache.org/viewvc/subversion/branches/ra-git/tools/dev/unix-build/Makefile.svn?rev=1717223&r1=1717222&r2=1717223&view=diff ============================================================================== --- subversion/branches/ra-git/tools/dev/unix-build/Makefile.svn (original) +++ subversion/branches/ra-git/tools/dev/unix-build/Makefile.svn Mon Nov 30 10:24:16 2015 @@ -79,7 +79,7 @@ SERF_OLD_VER = 0.3.1 CYRUS_SASL_VER = 2.1.25 SQLITE_VER = 3080500 LIBMAGIC_VER = 5.19 -RUBY_VER = 1.8.7-p358 +RUBY_VER = 1.8.7-p374 BZ2_VER = 1.0.6 PYTHON_VER = 2.7.8 JUNIT_VER = 4.10 @@ -109,7 +109,7 @@ SHA256_${NEON_DIST} = 2962cfcb5d30f3272e SHA256_${CYRUS_SASL_DIST} = 418c16e6240a4f9b637cbe3d62937b9675627bad27c622191d47de8686fe24fe SHA256_${SQLITE_DIST} = 98c33abe4106e508e73fda648b2657ac9e969fe24695f543dcde68cc71f3091b SHA256_${LIBMAGIC_DIST} = 9484b3bbda1acc7b13a4f71031a85ce10c77bd0ffec7226741a219ef587e3a7c -SHA256_${RUBY_DIST} = 9e0856d58830e08f1e38233947d859898ae09d4780cb1a502108e41308de33cb +SHA256_${RUBY_DIST} = 876eeeaaeeab10cbf4767833547d66d86d6717ef48fd3d89e27db8926a65276c SHA256_${BZ2_DIST} = a2848f34fcd5d6cf47def00461fcb528a0484d8edef8208d6d2e2909dc61d9cd SHA256_${PYTHON_DIST} = 74d70b914da4487aa1d97222b29e9554d042f825f26cb2b93abd20fdda56b557 SHA256_${JUNIT_DIST} = 36a747ca1e0b86f6ea88055b8723bb87030d627766da6288bf077afdeeb0f75a @@ -154,12 +154,12 @@ APR_UTIL_URL = https://svn.apache.org/re HTTPD_URL = https://archive.apache.org/dist/httpd/$(HTTPD_DIST) NEON_URL = http://webdav.org/neon/$(NEON_DIST) #SERF_URL = http://serf.googlecode.com/files/$(SERF_DIST) -SERF_URL = https://serf.googlecode.com/svn/tags/$(SERF_VER) -SERF_OLD_URL = https://serf.googlecode.com/svn/tags/$(SERF_OLD_VER) +SERF_URL = https://svn.apache.org/repos/asf/serf/tags/$(SERF_VER) +SERF_OLD_URL = https://svn.apache.org/repos/asf/serf/tags/$(SERF_OLD_VER) SQLITE_URL = https://www.sqlite.org/2014/$(SQLITE_DIST) CYRUS_SASL_URL = ftp://ftp.andrew.cmu.edu/pub/cyrus-mail/$(CYRUS_SASL_DIST) LIBMAGIC_URL = ftp://ftp.astron.com/pub/file/$(LIBMAGIC_DIST) -RUBY_URL = http://ftp.ruby-lang.org/pub/ruby/1.8/$(RUBY_DIST) +RUBY_URL = https://cache.ruby-lang.org/pub/ruby/1.8/$(RUBY_DIST) BZ2_URL = http://bzip.org/$(BZ2_VER)/$(BZ2_DIST) PYTHON_URL = https://python.org/ftp/python/$(PYTHON_VER)/$(PYTHON_DIST) JUNIT_URL = https://downloads.sourceforge.net/project/junit/junit/$(JUNIT_VER)/$(JUNIT_DIST) @@ -604,6 +604,8 @@ httpd-reset: httpd-clean: -(cd $(HTTPD_OBJDIR) && env MAKEFLAGS= make clean) + -rm ${HTTPD_OBJDIR}/chil-engine.diff + -rm ${HTTPD_OBJDIR}/ssl3.diff # fetch distfile for httpd $(DISTDIR)/$(HTTPD_DIST): @@ -627,14 +629,31 @@ $(HTTPD_OBJDIR)/chil-engine.diff: echo >>$@.tmp ' ap_log_error(APLOG_MARK, APLOG_ERR, 0, s,' mv -f $@.tmp $@ +$(HTTPD_OBJDIR)/ssl3.diff: + mkdir -p $(dir $@) + echo >$@.tmp '--- support/ab.c.orig Mon Sep 14 15:33:46 2015' + echo >>$@.tmp '+++ support/ab.c Mon Sep 14 15:34:15 2015' + echo >>$@.tmp '@@ -2232,8 +2232,10 @@' + echo >>$@.tmp ' } else if (strncasecmp(optarg, "SSL2", 4) == 0) {' + echo >>$@.tmp ' meth = SSLv2_client_method();' + echo >>$@.tmp ' #endif' + echo >>$@.tmp '+#ifndef OPENSSL_NO_SSL3' + echo >>$@.tmp ' } else if (strncasecmp(optarg, "SSL3", 4) == 0) {' + echo >>$@.tmp ' meth = SSLv3_client_method();' + echo >>$@.tmp '+#endif' + echo >>$@.tmp ' #ifdef HAVE_TLSV1_X' + echo >>$@.tmp ' } else if (strncasecmp(optarg, "TLS1.1", 6) == 0) {' + echo >>$@.tmp ' meth = TLSv1_1_client_method();' + mv -f $@.tmp $@ # retrieve httpd $(HTTPD_OBJDIR)/.retrieved: $(DISTDIR)/$(HTTPD_DIST) \ - $(HTTPD_OBJDIR)/chil-engine.diff + $(HTTPD_OBJDIR)/chil-engine.diff $(HTTPD_OBJDIR)/ssl3.diff $(call do_check_sha256,$(HTTPD_DIST)) [ -d $(HTTPD_OBJDIR) ] || mkdir -p $(HTTPD_OBJDIR) tar -C $(SRCDIR) -zxf $(DISTDIR)/$(HTTPD_DIST) cd $(HTTPD_SRCDIR) && patch -p0 < $(HTTPD_OBJDIR)/chil-engine.diff + cd $(HTTPD_SRCDIR) && patch -p0 < $(HTTPD_OBJDIR)/ssl3.diff cp $(HTTPD_SRCDIR)/modules/ssl/ssl_toolkit_compat.h \ $(HTTPD_SRCDIR)/modules/ssl/ssl_toolkit_compat.h.orig sed '/^#define HAVE_SSL_RAND_EGD/d' \ @@ -1060,6 +1079,11 @@ $(RUBY_OBJDIR)/.retrieved: $(DISTDIR)/$( $(call do_check_sha256,$(RUBY_DIST)) [ -d $(RUBY_OBJDIR) ] || mkdir -p $(RUBY_OBJDIR) tar -C $(SRCDIR) -zxf $(DISTDIR)/$(RUBY_DIST) + grep -v 'OSSL_SSL_METHOD_ENTRY(SSLv3' \ + $(RUBY_SRCDIR)/ext/openssl/ossl_ssl.c > \ + $(RUBY_SRCDIR)/ext/openssl/ossl_ssl.c.nossl3 + mv $(RUBY_SRCDIR)/ext/openssl/ossl_ssl.c.nossl3 \ + $(RUBY_SRCDIR)/ext/openssl/ossl_ssl.c touch $@ ifeq ($(THREADING),yes) @@ -1522,6 +1546,7 @@ $(SVN_OBJDIR)/.bindings-installed: $(SVN # run svn regression tests HTTPD_CHECK_CONF=$(PREFIX)/httpd/conf/httpd-svn-check-$(WC).conf HTTPD_CHECK_USERS=$(PREFIX)/httpd/conf/httpd-svn-check-users +HTTPD_CHECK_GROUPS=$(PREFIX)/httpd/conf/httpd-svn-check-groups HTTPD_CHECK_PORT=8081 MOD_DONTDOTHAT_CONF=$(PREFIX)/httpd/conf/dontdothat @@ -1531,13 +1556,18 @@ $(MOD_DONTDOTHAT_CONF): echo >>$@.tmp '/ = deny' mv -f $@.tmp $@ -$(HTTPD_CHECK_USERS): +$(HTTPD_CHECK_GROUPS): mkdir -p $(dir $@) - echo > $@.tmp 'jrandom:xCGl35kV9oWCY' - echo >>$@.tmp 'jconstant:xCGl35kV9oWCY' - mv -f $@.tmp $@ + printf "random: jrandom\nconstant: jconstant\n" > $@ -$(HTTPD_CHECK_CONF): $(HTTPD_CHECK_USERS) $(MOD_DONTDOTHAT_CONF) +$(HTTPD_CHECK_CONF): $(MOD_DONTDOTHAT_CONF) $(HTTPD_CHECK_GROUPS) + mkdir -p $(dir $@) + $(PREFIX)/httpd/bin/htpasswd -bc $(HTTPD_CHECK_USERS).tmp jrandom rayjandom + $(PREFIX)/httpd/bin/htpasswd -b $(HTTPD_CHECK_USERS).tmp jconstant rayjandom + $(PREFIX)/httpd/bin/htpasswd -b $(HTTPD_CHECK_USERS).tmp __dumpster__ __loadster__ + $(PREFIX)/httpd/bin/htpasswd -b $(HTTPD_CHECK_USERS).tmp JRANDOM rayjandom + $(PREFIX)/httpd/bin/htpasswd -b $(HTTPD_CHECK_USERS).tmp JCONSTANT rayjandom + mv -f $(HTTPD_CHECK_USERS).tmp $(HTTPD_CHECK_USERS) echo > $@.tmp '# httpd config for make check' echo >>$@.tmp 'ServerRoot "$(PREFIX)/httpd"' echo >>$@.tmp 'Listen localhost:$(HTTPD_CHECK_PORT)' @@ -1601,8 +1631,8 @@ endif echo >>$@.tmp '# Location for tests using mod_dontdothat' echo >>$@.tmp '' echo >> $@.tmp 'DAV svn' - echo >> $@.tmp 'SVNParentPath "$(SVN_WC)/subversion/tests/cmdline/svn-test-work/repositories"' - echo >> $@.tmp 'AuthzSVNAccessFile "$(SVN_WC)/subversion/tests/cmdline/svn-test-work/authz"' + echo >> $@.tmp 'SVNParentPath $(SVN_WC)/subversion/tests/cmdline/svn-test-work/repositories' + echo >> $@.tmp 'AuthzSVNAccessFile $(SVN_WC)/subversion/tests/cmdline/svn-test-work/authz' echo >> $@.tmp 'AuthType Basic' echo >> $@.tmp 'AuthName "Subversion Repository"' echo >> $@.tmp 'AuthzSVNAccessFile $(SVN_WC)/subversion/tests/cmdline/svn-test-work/authz' @@ -1613,6 +1643,169 @@ ifeq ($(USE_HTTPV1),yes) endif echo >> $@.tmp 'DontDoThatConfigFile "$(MOD_DONTDOTHAT_CONF)"' echo >> $@.tmp '' + + echo >>$@.tmp '# Several locations for mod_authz_svn test follow' + echo >>$@.tmp '' + echo >>$@.tmp ' DAV svn' + echo >>$@.tmp ' SVNParentPath $(SVN_WC)/subversion/tests/cmdline/svn-test-work/local_tmp' + echo >>$@.tmp ' AuthzSVNAccessFile $(SVN_WC)/subversion/tests/cmdline/svn-test-work/authz' +ifeq ($(USE_HTTPV1),yes) + echo >>$@.tmp ' SVNAdvertiseV2Protocol off' +endif + echo >>$@.tmp ' SVNListParentPath On' + echo >>$@.tmp ' ' + echo >>$@.tmp ' Require all granted' + echo >>$@.tmp ' ' + echo >>$@.tmp ' ' + echo >>$@.tmp ' Allow from all' + echo >>$@.tmp ' ' +ifeq ($(USE_AUTHZ_SHORT_CIRCUIT),yes) + echo >>$@.tmp ' SVNPathAuthz short_circuit' +endif + echo >>$@.tmp '' + echo >>$@.tmp '' + echo >>$@.tmp ' DAV svn' + echo >>$@.tmp ' SVNParentPath $(SVN_WC)/subversion/tests/cmdline/svn-test-work/local_tmp' + echo >>$@.tmp ' AuthzSVNAccessFile $(SVN_WC)/subversion/tests/cmdline/svn-test-work/authz' +ifeq ($(USE_HTTPV1),yes) + echo >>$@.tmp ' SVNAdvertiseV2Protocol off' +endif + echo >>$@.tmp ' SVNListParentPath On' + echo >>$@.tmp ' AuthType Basic' + echo >>$@.tmp ' AuthName "Subversion Repository"' + echo >>$@.tmp ' AuthUserFile $(HTTPD_CHECK_USERS)' + echo >>$@.tmp ' Require valid-user' + echo >>$@.tmp ' Satisfy Any' +ifeq ($(USE_AUTHZ_SHORT_CIRCUIT),yes) + echo >>$@.tmp ' SVNPathAuthz short_circuit' +endif + echo >>$@.tmp '' + echo >>$@.tmp '' + echo >>$@.tmp ' DAV svn' + echo >>$@.tmp ' SVNParentPath $(SVN_WC)/subversion/tests/cmdline/svn-test-work/local_tmp' + echo >>$@.tmp ' AuthzSVNAccessFile $(SVN_WC)/subversion/tests/cmdline/svn-test-work/authz' +ifeq ($(USE_HTTPV1),yes) + echo >>$@.tmp ' SVNAdvertiseV2Protocol off' +endif + echo >>$@.tmp ' SVNListParentPath On' + echo >>$@.tmp ' AuthType Basic' + echo >>$@.tmp ' AuthName "Subversion Repository"' + echo >>$@.tmp ' AuthUserFile $(HTTPD_CHECK_USERS)' + echo >>$@.tmp ' Require valid-user' + echo >>$@.tmp ' AuthzSVNNoAuthWhenAnonymousAllowed On' + echo >>$@.tmp ' SVNPathAuthz On' +ifeq ($(USE_AUTHZ_SHORT_CIRCUIT),yes) + echo >>$@.tmp ' SVNPathAuthz short_circuit' +endif + echo >>$@.tmp '' + echo >>$@.tmp '' + echo >>$@.tmp ' DAV svn' + echo >>$@.tmp ' SVNParentPath $(SVN_WC)/subversion/tests/cmdline/svn-test-work/local_tmp' + echo >>$@.tmp ' AuthzSVNAccessFile $(SVN_WC)/subversion/tests/cmdline/svn-test-work/authz' +ifeq ($(USE_HTTPV1),yes) + echo >>$@.tmp ' SVNAdvertiseV2Protocol off' +endif + echo >>$@.tmp ' SVNListParentPath On' + echo >>$@.tmp ' AuthType Basic' + echo >>$@.tmp ' AuthName "Subversion Repository"' + echo >>$@.tmp ' AuthUserFile $(HTTPD_CHECK_USERS)' + echo >>$@.tmp ' Require valid-user' +ifeq ($(USE_AUTHZ_SHORT_CIRCUIT),yes) + echo >>$@.tmp ' SVNPathAuthz short_circuit' +endif + echo >>$@.tmp '' + echo >>$@.tmp '' + echo >>$@.tmp ' DAV svn' + echo >>$@.tmp ' SVNParentPath $(SVN_WC)/subversion/tests/cmdline/svn-test-work/local_tmp' + echo >>$@.tmp ' AuthzSVNAccessFile $(SVN_WC)/subversion/tests/cmdline/svn-test-work/authz' +ifeq ($(USE_HTTPV1),yes) + echo >>$@.tmp ' SVNAdvertiseV2Protocol off' +endif + echo >>$@.tmp ' SVNListParentPath On' + echo >>$@.tmp ' AuthType Basic' + echo >>$@.tmp ' AuthName "Subversion Repository"' + echo >>$@.tmp ' AuthUserFile $(HTTPD_CHECK_USERS)' + echo >>$@.tmp ' Require valid-user' + echo >>$@.tmp ' AuthzSVNAnonymous Off' + echo >>$@.tmp ' SVNPathAuthz On' + echo >>$@.tmp '' + echo >>$@.tmp '' + echo >>$@.tmp ' DAV svn' + echo >>$@.tmp ' SVNParentPath $(SVN_WC)/subversion/tests/cmdline/svn-test-work/local_tmp' + echo >>$@.tmp ' AuthzSVNAccessFile $(SVN_WC)/subversion/tests/cmdline/svn-test-work/authz' +ifeq ($(USE_HTTPV1),yes) + echo >>$@.tmp ' SVNAdvertiseV2Protocol off' +endif + echo >>$@.tmp ' SVNListParentPath On' + echo >>$@.tmp ' AuthType Basic' + echo >>$@.tmp ' AuthName "Subversion Repository"' + echo >>$@.tmp ' AuthUserFile $(HTTPD_CHECK_USERS)' + echo >>$@.tmp ' Require valid-user' + echo >>$@.tmp ' AuthzForceUsernameCase Lower' +ifeq ($(USE_AUTHZ_SHORT_CIRCUIT),yes) + echo >>$@.tmp ' SVNPathAuthz short_circuit' +endif + echo >>$@.tmp '' + echo >>$@.tmp '' + echo >>$@.tmp ' DAV svn' + echo >>$@.tmp ' SVNParentPath $(SVN_WC)/subversion/tests/cmdline/svn-test-work/local_tmp' + echo >>$@.tmp ' AuthzSVNAccessFile $(SVN_WC)/subversion/tests/cmdline/svn-test-work/authz' +ifeq ($(USE_HTTPV1),yes) + echo >>$@.tmp ' SVNAdvertiseV2Protocol off' +endif + echo >>$@.tmp ' SVNListParentPath On' + echo >>$@.tmp ' AuthType Basic' + echo >>$@.tmp ' AuthName "Subversion Repository"' + echo >>$@.tmp ' AuthUserFile $(HTTPD_CHECK_USERS)' + echo >>$@.tmp ' AuthGroupFile $(HTTPD_CHECK_GROUPS)' + echo >>$@.tmp ' Require group random' + echo >>$@.tmp ' AuthzSVNAuthoritative Off' + echo >>$@.tmp ' SVNPathAuthz On' + echo >>$@.tmp '' + echo >>$@.tmp '' + echo >>$@.tmp ' ' + echo >>$@.tmp ' DAV svn' + echo >>$@.tmp ' SVNParentPath $($SVN_WC)/subversion/tests/cmdline/svn-test-work/local_tmp' + echo >>$@.tmp ' AuthzSVNAccessFile $(SVN_WC)/subversion/tests/cmdline/svn-test-work/authz' +ifeq ($(USE_HTTPV1),yes) + echo >>$@.tmp ' SVNAdvertiseV2Protocol off' +endif + echo >>$@.tmp ' SVNListParentPath On' + echo >>$@.tmp ' AuthType Basic' + echo >>$@.tmp ' AuthName "Subversion Repository"' + echo >>$@.tmp ' AuthUserFile $(HTTPD_CHECK_USERS)' + echo >>$@.tmp ' AuthzSendForbiddenOnFailure On' + echo >>$@.tmp ' Satisfy All' + echo >>$@.tmp ' ' + echo >>$@.tmp ' Require valid-user' + echo >>$@.tmp ' Require expr req("ALLOW") == "1"' + echo >>$@.tmp ' ' +ifeq ($(USE_AUTHZ_SHORT_CIRCUIT),yes) + echo >>$@.tmp ' SVNPathAuthz short_circuit' +endif + echo >>$@.tmp ' ' + echo >>$@.tmp ' ' + echo >>$@.tmp ' DAV svn' + echo >>$@.tmp ' SVNParentPath $(SVN_WC)/subversion/tests/cmdline/svn-test-work/local_tmp' + echo >>$@.tmp ' AuthzSVNAccessFile $(SVN_WC)/subversion/tests/cmdline/svn-test-work/authz' +ifeq ($(USE_HTTPV1),yes) + echo >>$@.tmp ' SVNAdvertiseV2Protocol off' +endif + echo >>$@.tmp ' SVNListParentPath On' + echo >>$@.tmp ' AuthType Basic' + echo >>$@.tmp ' AuthName "Subversion Repository"' + echo >>$@.tmp ' AuthUserFile $(HTTPD_CHECK_USERS)' + echo >>$@.tmp ' AuthzSendForbiddenOnFailure On' + echo >>$@.tmp ' Satisfy All' + echo >>$@.tmp ' ' + echo >>$@.tmp ' Require valid-user' + echo >>$@.tmp ' Require expr req("ALLOW") == "1"' + echo >>$@.tmp ' ' +ifeq ($(USE_AUTHZ_SHORT_CIRCUIT),yes) + echo >>$@.tmp ' SVNPathAuthz short_circuit' +endif + echo >>$@.tmp ' ' + echo >>$@.tmp '' echo >>$@.tmp 'RedirectMatch permanent ^/svn-test-work/repositories/REDIRECT-PERM-(.*)$$ /svn-test-work/repositories/$$1' echo >>$@.tmp 'RedirectMatch ^/svn-test-work/repositories/REDIRECT-TEMP-(.*)$$ /svn-test-work/repositories/$$1' echo >>$@.tmp 'Include "conf/$(SVN_REL_WC)*-custom.conf"' Propchange: subversion/branches/ra-git/tools/dev/wc-ng/ ------------------------------------------------------------------------------ --- svn:ignore (added) +++ svn:ignore Mon Nov 30 10:24:16 2015 @@ -0,0 +1 @@ +svn-wc-db-tester Modified: subversion/branches/ra-git/tools/dev/x509-parser.c URL: http://svn.apache.org/viewvc/subversion/branches/ra-git/tools/dev/x509-parser.c?rev=1717223&r1=1717222&r2=1717223&view=diff ============================================================================== --- subversion/branches/ra-git/tools/dev/x509-parser.c (original) +++ subversion/branches/ra-git/tools/dev/x509-parser.c Mon Nov 30 10:24:16 2015 @@ -94,7 +94,8 @@ get_der_cert_from_stream(const svn_strin apr_pool_t *pool) { svn_string_t *raw; - SVN_ERR(svn_string_from_stream(&raw, in, pool, pool)); + SVN_ERR(svn_string_from_stream2(&raw, in, SVN__STREAM_CHUNK_SIZE, + pool)); *der_cert = NULL; @@ -158,7 +159,7 @@ int main (int argc, const char *argv[]) } else if (argc == 1) { - err = svn_stream_for_stdin(&in, pool); + err = svn_stream_for_stdin2(&in, TRUE, pool); } else err = svn_error_create(SVN_ERR_CL_ARG_PARSING_ERROR, NULL, _("Too many arguments")); Modified: subversion/branches/ra-git/tools/dist/backport.pl URL: http://svn.apache.org/viewvc/subversion/branches/ra-git/tools/dist/backport.pl?rev=1717223&r1=1717222&r2=1717223&view=diff ============================================================================== --- subversion/branches/ra-git/tools/dist/backport.pl (original) +++ subversion/branches/ra-git/tools/dist/backport.pl Mon Nov 30 10:24:16 2015 @@ -11,8 +11,10 @@ use v5.10.0; # needed for $^V # # As of Perl v5.20.1, the semantics of given/when provided by Perl are # compatible with those expected by the script, so disable the warning for -# those Perls. -no if $^V le v5.20.1, warnings => 'experimental::smartmatch'; +# those Perls. But don't try to disable the the warning category on Perls +# that don't know that category, since that breaks compilation. +no if (v5.17.0 le $^V and $^V le v5.20.1), + warnings => 'experimental::smartmatch'; # Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file @@ -133,6 +135,39 @@ $SVNq = "$SVN -q "; $SVNq =~ s/-q// if $DEBUG; +my $BACKPORT_OPTIONS_HELP = <{entry}->{header}." } @votesarray; (@sentences == 1) - ? $sentences[0] + ? "* STATUS: $sentences[0]" : "* STATUS:\n" . join "", map " $_\n", @sentences; }; @@ -1015,13 +1040,13 @@ sub handle_entry { # See above for why the while(1). QUESTION: while (1) { my $key = $entry{digest}; - given (prompt 'Run a merge? [y,l,v,±1,±0,q,e,a, ,N] ', + given (prompt 'Run a merge? [y,l,v,±1,±0,q,e,a, ,N,?] ', verbose => 1, extra => qr/[+-]/) { when (/^y/i) { - #validate_branch_contains_named_revisions %entry; + # TODO: validate_branch_contains_named_revisions %entry; merge \%entry; while (1) { - given (prompt "Shall I open a subshell? [ydN] ", verbose => 1) { + given (prompt "Shall I open a subshell? [ydN?] ", verbose => 1) { when (/^y/i) { # TODO: if $MAY_COMMIT, save the log message to a file (say, # backport.logmsg in the wcroot). @@ -1033,6 +1058,10 @@ sub handle_entry { or warn "diff failed ($?): $!"; next; } + when (/^[?]/i) { + print $BACKPORT_OPTIONS_MERGE_OPTIONS_HELP; + next; + } when (/^N/i) { # fall through. } @@ -1096,6 +1125,10 @@ sub handle_entry { when (/^\x20/) { last PROMPT; # Fall off the end of the given/when block. } + when (/^[?]/i) { + print $BACKPORT_OPTIONS_HELP; + next QUESTION; + } default { say "Please use one of the options in brackets (q to quit)!"; next QUESTION; @@ -1231,7 +1264,7 @@ sub nominate_main { } my @lines; - warn "Wrapping [$logmsg]\n"; + warn "Wrapping [$logmsg]\n" if $DEBUG; push @lines, wrap " * ", ' 'x3, join ', ', map "r$_", @revnums; push @lines, wrap ' 'x3, ' 'x3, split /\n/, $logmsg; push @lines, " Justification:"; @@ -1263,7 +1296,7 @@ sub nominate_main { # Done! system "$SVN diff -- $STATUS"; if (prompt "Commit this nomination? ") { - system "$SVN commit -m 'Nominate r$revnums[0].' -- $STATUS"; + system "$SVN commit -m '* STATUS: Nominate r$revnums[0].' -- $STATUS"; exit $?; } elsif (!$had_local_mods or prompt "Revert STATUS (destroying local mods)? ") { Modified: subversion/branches/ra-git/tools/dist/backport_tests.py URL: http://svn.apache.org/viewvc/subversion/branches/ra-git/tools/dist/backport_tests.py?rev=1717223&r1=1717222&r2=1717223&view=diff ============================================================================== --- subversion/branches/ra-git/tools/dist/backport_tests.py (original) +++ subversion/branches/ra-git/tools/dist/backport_tests.py Mon Nov 30 10:24:16 2015 @@ -1,7 +1,7 @@ #!/usr/bin/env python # py:encoding=utf-8 # -# backport_tests.py: Test backport.pl +# backport_tests.py: Test backport.pl or backport.py # # Subversion is a tool for revision control. # See http://subversion.apache.org for more information. @@ -25,6 +25,25 @@ # under the License. ###################################################################### +# We'd like to test backport.pl and backport.py the same way, and to reuse +# the svntest Python harness. Since the latter standardizes argv parsing, +# we can't use argv to determine whether .py or .pl should be tested. Thus, +# we implement the tests themselves in this file, while two driver files +# (backport_tests_pl.py and backport_tests_py.py) invoke this file set +# to run either backport-suite implementation. +# +# ### Note: the two driver scripts use the same repository names in +# ### svn-test-work. This is not ideal, but hopefully acceptable +# ### temporarily until we switch over to backport.py and remove backport.pl. +# ### +# ### See svntest.testcase.FunctionTestCase.get_sandbox_name(). +try: + run_backport, run_conflicter +except NameError: + raise Exception("Failure: %s should not be run directly, or the wrapper " + "does not define both run_backport() and run_conflicter()" + % __file__) + # General modules import contextlib import functools @@ -58,8 +77,6 @@ Wimp = svntest.testcase.Wimp_deco ###################################################################### # Helper functions -BACKPORT_PL = os.path.abspath(os.path.join(os.path.dirname(__file__), - 'backport.pl')) STATUS = 'branch/STATUS' class BackportTest(object): @@ -159,12 +176,15 @@ def serialize_entry(entry): ]) def serialize_STATUS(approveds, + candidates=[], serialize_entry=serialize_entry): """Construct and return the contents of a STATUS file. APPROVEDS is an iterable of ENTRY dicts. The dicts are defined to have the following keys: 'revisions', a list of revision numbers (ints); 'logsummary'; and 'votes', a dict mapping ±1/±0 (int) to list of voters. + + CANDIDATES is like APPROVEDS, except added to a different section of the file. """ strings = [] @@ -173,6 +193,8 @@ def serialize_STATUS(approveds, strings.append("Candidate changes:\n") strings.append("==================\n\n") + strings.extend(map(serialize_entry, candidates)) + strings.append("Random new subheading:\n") strings.append("======================\n\n") @@ -186,22 +208,6 @@ def serialize_STATUS(approveds, return "".join(strings) -def run_backport(sbox, error_expected=False, extra_env=[]): - """Run backport.pl. EXTRA_ENV is a list of key=value pairs (str) to set in - the child's environment. ERROR_EXPECTED is propagated to run_command().""" - # TODO: if the test is run in verbose mode, pass DEBUG=1 in the environment, - # and pass error_expected=True to run_command() to not croak on - # stderr output from the child (because it uses 'sh -x'). - args = [ - '/usr/bin/env', - 'SVN=' + svntest.main.svn_binary, - 'YES=1', 'MAY_COMMIT=1', 'AVAILID=jrandom', - ] + list(extra_env) + [ - 'perl', BACKPORT_PL, - ] - with chdir(sbox.ospath('branch')): - return svntest.main.run_command(args[0], error_expected, False, *(args[1:])) - def verify_backport(sbox, expected_dump_file, uuid): """Compare the contents of the SBOX repository with EXPECTED_DUMP_FILE. Set the UUID of SBOX to UUID beforehand. @@ -225,7 +231,7 @@ def verify_backport(sbox, expected_dump_ src_dump = svntest.actions.run_and_verify_dump(sbox.repo_dir) svntest.verify.compare_dump_files( - "Dump files", "DUMP", src_dump, dest_dump) + "Dump files", "DUMP", dest_dump, src_dump) ###################################################################### # Tests @@ -376,16 +382,14 @@ def backport_conflicts_detection(sbox): sbox.simple_commit(message="Conflicting change on iota") # r7: nominate r4, but without the requisite --accept - approved_entries = [ + candidate_entries = [ make_entry([4], notes="This will conflict."), ] - sbox.simple_append(STATUS, serialize_STATUS(approved_entries)) + sbox.simple_append(STATUS, serialize_STATUS([], candidate_entries)) sbox.simple_commit(message='Nominate r4') # Run it. - exit_code, output, errput = run_backport(sbox, True, - # Choose conflicts mode: - ["MAY_COMMIT=0"]) + exit_code, output, errput = run_conflicter(sbox, True) # Verify the conflict is detected. expected_output = svntest.verify.RegexOutput( @@ -418,9 +422,9 @@ def backport_conflicts_detection(sbox): sbox.simple_commit(message='Re-nominate r4') # Detect conflicts. - exit_code, output, errput = run_backport(sbox, extra_env=["MAY_COMMIT=0"]) + exit_code, output, errput = run_conflicter(sbox) - # Verify stdout. (exit_code and errput were verified by run_backport().) + # Verify stdout. (exit_code and errput were verified by run_conflicter().) svntest.verify.verify_outputs(None, output, errput, "Conflicts found.*, as expected.", []) @@ -501,7 +505,7 @@ def backport_double_conflict(sbox): sbox.simple_commit(message='Nominate the r4 group') # Run it, in conflicts mode. - exit_code, output, errput = run_backport(sbox, True, ["MAY_COMMIT=0"]) + exit_code, output, errput = run_conflicter(sbox, True) # Verify the failure mode: "merge conflict" error on stderr, but backport.pl # itself exits with code 0, since conflicts were confined to Depends:-ed @@ -541,13 +545,16 @@ def backport_double_conflict(sbox): sbox.simple_append(STATUS, serialize_STATUS(approved_entries), truncate=True) sbox.simple_commit(message='Re-nominate the r4 group') - exit_code, output, errput = run_backport(sbox, True, ["MAY_COMMIT=0"]) + exit_code, output, errput = run_conflicter(sbox, True) + ## An unexpected non-zero exit code is treated as a fatal error. # [1-9]\d+ matches non-zero exit codes - expected_errput = r'r4 .*: subshell exited with code (?:[1-9]\d+)' + expected_stdout = None + expected_errput = r'r4 .*: subshell exited with code (?:[1-9]\d+)' \ + r"|.*subprocess.CalledProcessError.*'merge'.*exit status 1" svntest.verify.verify_exit_code(None, exit_code, 1) svntest.verify.verify_outputs(None, output, errput, - svntest.verify.AnyOutput, expected_errput) + expected_stdout, expected_errput) @@ -590,6 +597,55 @@ def backport_branch_with_original_revisi #---------------------------------------------------------------------- +@BackportTest(None) +def backport_otherproject_change(sbox): + "inoperative revision" + + # r6: a change outside ^/subversion + sbox.simple_mkdir('elsewhere') + sbox.simple_commit() + + # r7: Nominate r6 by mistake + approved_entries = [ + make_entry([6]) + ] + sbox.simple_append(STATUS, serialize_STATUS(approved_entries)) + sbox.simple_commit(message='Nominate r6 by mistake') + + # Run it. + exit_code, output, errput = run_backport(sbox, error_expected=True) + + # Verify no commit occurred. + svntest.actions.run_and_verify_svnlook(["7\n"], [], + 'youngest', sbox.repo_dir) + + # Verify the failure mode. + expected_stdout = None + expected_stderr = ".*only svn:mergeinfo changes.*" + if exit_code == 0: + # Can't use verify_exit_code() since the exact code used varies. + raise svntest.Failure("exit_code should be non-zero") + svntest.verify.verify_outputs(None, output, errput, + expected_stdout, expected_stderr) + +#---------------------------------------------------------------------- +@BackportTest(None) +def backport_STATUS_mods(sbox): + "local mods to STATUS" + + # Introduce a local mod. + sbox.simple_append(STATUS, "\n") + + exit_code, output, errput = run_backport(sbox, error_expected=True) + expected_stdout = None + expected_stderr = ".*Local mods.*STATUS.*" + if exit_code == 0: + # Can't use verify_exit_code() since the exact code used varies. + raise svntest.Failure("exit_code should be non-zero") + svntest.verify.verify_outputs(None, output, errput, + expected_stdout, expected_stderr) + +#---------------------------------------------------------------------- ######################################################################## # Run the tests @@ -605,6 +661,8 @@ test_list = [ None, backport_branch_contains, backport_double_conflict, backport_branch_with_original_revision, + backport_otherproject_change, + backport_STATUS_mods, # When adding a new test, include the test number in the last # 6 bytes of the UUID. ] Propchange: subversion/branches/ra-git/tools/dist/backport_tests.py ('svn:executable' removed) Modified: subversion/branches/ra-git/tools/dist/rat-excludes URL: http://svn.apache.org/viewvc/subversion/branches/ra-git/tools/dist/rat-excludes?rev=1717223&r1=1717222&r2=1717223&view=diff ============================================================================== --- subversion/branches/ra-git/tools/dist/rat-excludes (original) +++ subversion/branches/ra-git/tools/dist/rat-excludes Mon Nov 30 10:24:16 2015 @@ -32,7 +32,7 @@ subversion/bindings/ctypes-python/csvn/e subversion/tests/cmdline/svntest/err.py tools/buildbot/master/public_html/buildbot.css tools/dist/rat-excludes -tools/dist/_gnupg.py +tools/dist/security/_gnupg.py tools/dist/templates/*.ezt tools/dev/iz/defect.dem tools/dev/iz/ff2csv.command Modified: subversion/branches/ra-git/tools/dist/release.py URL: http://svn.apache.org/viewvc/subversion/branches/ra-git/tools/dist/release.py?rev=1717223&r1=1717222&r2=1717223&view=diff ============================================================================== --- subversion/branches/ra-git/tools/dist/release.py (original) +++ subversion/branches/ra-git/tools/dist/release.py Mon Nov 30 10:24:16 2015 @@ -34,9 +34,6 @@ # It'd be kind of nice to use the Subversion python bindings in this script, # but people.apache.org doesn't currently have them installed -# Futures (Python 2.5 compatibility) -from __future__ import with_statement - # Stuff we need import os import re @@ -86,13 +83,13 @@ except AttributeError: tool_versions = { 'trunk' : { 'autoconf' : '2.69', - 'libtool' : '2.4.3', - 'swig' : '3.0.0', + 'libtool' : '2.4.6', + 'swig' : '2.0.12', }, '1.9' : { 'autoconf' : '2.69', - 'libtool' : '2.4.3', - 'swig' : '3.0.0' + 'libtool' : '2.4.6', + 'swig' : '2.0.12' }, '1.8' : { 'autoconf' : '2.69', @@ -356,6 +353,13 @@ class LibtoolDep(RollDep): # system libtool (I'm looking at you, Debian). return False + def build(self): + RollDep.build(self) + # autogen.sh looks for glibtoolize before libtoolize + bin_dir = os.path.join(get_prefix(self._base_dir), "bin") + os.symlink("libtoolize", os.path.join(bin_dir, "glibtoolize")) + os.symlink("libtool", os.path.join(bin_dir, "glibtool")) + class SwigDep(RollDep): def __init__(self, base_dir, use_existing, verbose, swig_ver, sf_mirror): @@ -421,6 +425,29 @@ def compare_changes(repos, branch, revis logging.warning('CHANGES has unmerged revisions: %s' % stdout.replace("\n", " ")) + +_current_year = str(datetime.datetime.now().year) +_copyright_re = re.compile(r'Copyright (?:\(C\) )?(?P[0-9]+)' + r' The Apache Software Foundation', + re.MULTILINE) + +def check_copyright_year(repos, branch, revision): + def check_file(branch_relpath): + file_url = (repos + '/' + branch + '/' + + branch_relpath + '@' + str(revision)) + cat_cmd = ['svn', 'cat', file_url] + stdout = subprocess.check_output(cat_cmd) + m = _copyright_re.search(stdout) + if m: + year = m.group('year') + else: + year = None + if year != _current_year: + logging.warning('Copyright year in ' + branch_relpath + + ' is not the current year') + check_file('NOTICE') + check_file('subversion/libsvn_subr/version.c') + def roll_tarballs(args): 'Create the release artifacts.' @@ -432,6 +459,8 @@ def roll_tarballs(args): logging.info('Rolling release %s from branch %s@%d' % (args.version, branch, args.revnum)) + check_copyright_year(repos, args.branch, args.revnum) + # Ensure we've got the appropriate rolling dependencies available autoconf = AutoconfDep(args.base_dir, False, args.verbose, tool_versions[args.version.branch]['autoconf']) @@ -774,7 +803,7 @@ def get_siginfo(args, quiet=False): try: import gnupg except ImportError: - import _gnupg as gnupg + import security._gnupg as gnupg gpg = gnupg.GPG() target = get_target(args) Modified: subversion/branches/ra-git/tools/dist/templates/rc-news.ezt URL: http://svn.apache.org/viewvc/subversion/branches/ra-git/tools/dist/templates/rc-news.ezt?rev=1717223&r1=1717222&r2=1717223&view=diff ============================================================================== --- subversion/branches/ra-git/tools/dist/templates/rc-news.ezt (original) +++ subversion/branches/ra-git/tools/dist/templates/rc-news.ezt Mon Nov 30 10:24:16 2015 @@ -16,7 +16,7 @@ in the [version_base] release.

To get this release from the nearest mirror, please visit our - download page.

+ download page.

Modified: subversion/branches/ra-git/tools/dist/templates/rc-release-ann.ezt URL: http://svn.apache.org/viewvc/subversion/branches/ra-git/tools/dist/templates/rc-release-ann.ezt?rev=1717223&r1=1717222&r2=1717223&view=diff ============================================================================== --- subversion/branches/ra-git/tools/dist/templates/rc-release-ann.ezt (original) +++ subversion/branches/ra-git/tools/dist/templates/rc-release-ann.ezt Mon Nov 30 10:24:16 2015 @@ -1,7 +1,7 @@ I'm happy to announce the release of Apache Subversion [version]. Please choose the mirror closest to you by visiting: - http://subversion.apache.org/download/#[anchor] + http://subversion.apache.org/download.cgi#[anchor] The SHA1 checksums are: Modified: subversion/branches/ra-git/tools/dist/templates/stable-news.ezt URL: http://svn.apache.org/viewvc/subversion/branches/ra-git/tools/dist/templates/stable-news.ezt?rev=1717223&r1=1717222&r2=1717223&view=diff ============================================================================== --- subversion/branches/ra-git/tools/dist/templates/stable-news.ezt (original) +++ subversion/branches/ra-git/tools/dist/templates/stable-news.ezt Mon Nov 30 10:24:16 2015 @@ -13,7 +13,7 @@ >change log for more information about this release.

To get this release from the nearest mirror, please visit our - download page.

+ download page.

Modified: subversion/branches/ra-git/tools/dist/templates/stable-release-ann.ezt URL: http://svn.apache.org/viewvc/subversion/branches/ra-git/tools/dist/templates/stable-release-ann.ezt?rev=1717223&r1=1717222&r2=1717223&view=diff ============================================================================== --- subversion/branches/ra-git/tools/dist/templates/stable-release-ann.ezt (original) +++ subversion/branches/ra-git/tools/dist/templates/stable-release-ann.ezt Mon Nov 30 10:24:16 2015 @@ -1,7 +1,7 @@ I'm happy to announce the release of Apache Subversion [version]. Please choose the mirror closest to you by visiting: - http://subversion.apache.org/download/#[anchor] + http://subversion.apache.org/download.cgi#[anchor] The SHA1 checksums are: Modified: subversion/branches/ra-git/win-tests.py URL: http://svn.apache.org/viewvc/subversion/branches/ra-git/win-tests.py?rev=1717223&r1=1717222&r2=1717223&view=diff ============================================================================== --- subversion/branches/ra-git/win-tests.py (original) +++ subversion/branches/ra-git/win-tests.py Mon Nov 30 10:24:16 2015 @@ -31,6 +31,8 @@ import os, sys, subprocess import filecmp import shutil import traceback +import logging +import re try: # Python >=3.0 import configparser @@ -59,7 +61,6 @@ def _usage_exit(): print(" -u URL, --url=URL : run ra_dav or ra_svn tests against URL;") print(" will start svnserve for ra_svn tests") print(" -v, --verbose : talk more") - print(" -q, --quiet : talk less") print(" -f, --fs-type=type : filesystem type to use (fsfs is default)") print(" -c, --cleanup : cleanup after running a test") print(" -t, --test=TEST : Run the TEST test (all is default); use") @@ -83,6 +84,16 @@ def _usage_exit(): print(" --disable-http-v2 : Do not advertise support for HTTPv2 on server") print(" --disable-bulk-updates : Disable bulk updates on HTTP server") print(" --ssl-cert : Path to SSL server certificate to trust.") + print(" --https : Run Apache httpd with an https setup.") + print(" --http2 : Enable http2 in Apache Httpd (>= 2.4.17).") + print(" --global-scheduler : Enable global scheduler.") + print(" --exclusive-wc-locks : Enable exclusive working copy locks") + print(" --memcached-dir=DIR : Run memcached from dir") + print(" --memcached-server= : Enable usage of the specified memcached server") + print(" ") + print(" --skip-c-tests : Skip all C tests") + print(" --dump-load-cross-check: Run the dump load cross check after every test") + print(" --javahl : Run the javahl tests instead of the normal tests") print(" --swig=language : Run the swig perl/python/ruby tests instead of") print(" the normal tests") @@ -101,6 +112,8 @@ def _usage_exit(): print(" --config-file : Configuration file for tests") print(" --fsfs-sharding : Specify shard size (for fsfs)") print(" --fsfs-packing : Run 'svnadmin pack' automatically") + print(" -q, --quiet : Deprecated; this is the default.") + print(" Use --set-log-level instead.") sys.exit(0) @@ -120,19 +133,23 @@ gen_obj = gen_win_dependencies.GenDepend opts, args = my_getopt(sys.argv[1:], 'hrdvqct:pu:f:', ['release', 'debug', 'verbose', 'quiet', 'cleanup', 'test=', 'url=', 'svnserve-args=', 'fs-type=', 'asp.net-hack', - 'httpd-dir=', 'httpd-port=', 'httpd-daemon', + 'httpd-dir=', 'httpd-port=', 'httpd-daemon', 'https', 'httpd-server', 'http-short-circuit', 'httpd-no-log', 'disable-http-v2', 'disable-bulk-updates', 'help', 'fsfs-packing', 'fsfs-sharding=', 'javahl', 'swig=', - 'list', 'enable-sasl', 'bin=', 'parallel', + 'list', 'enable-sasl', 'bin=', 'parallel', 'http2', + 'global-scheduler', 'config-file=', 'server-minor-version=', 'log-level=', 'log-to-stdout', 'mode-filter=', 'milestone-filter=', - 'ssl-cert=']) + 'ssl-cert=', 'exclusive-wc-locks', 'memcached-server=', + 'skip-c-tests', 'dump-load-cross-check', 'memcached-dir=', + ]) if len(args) > 1: print('Warning: non-option arguments after the first one will be ignored') # Interpret the options and set parameters -base_url, fs_type, verbose, quiet, cleanup = None, None, None, None, None +base_url, fs_type, verbose, cleanup = None, None, None, None +global_scheduler = None repo_loc = 'local repository.' objdir = 'Debug' log = 'tests.log' @@ -143,6 +160,8 @@ run_httpd = None httpd_port = None httpd_service = None httpd_no_log = None +use_ssl = False +use_http2 = False http_short_circuit = False advertise_httpv2 = True http_bulk_updates = True @@ -162,6 +181,12 @@ mode_filter=None tests_to_run = [] log_level = None ssl_cert = None +exclusive_wc_locks = None +run_memcached = None +memcached_server = None +memcached_dir = None +skip_c_tests = None +dump_load_cross_check = None for opt, val in opts: if opt in ('-h', '--help'): @@ -172,8 +197,7 @@ for opt, val in opts: fs_type = val elif opt in ('-v', '--verbose'): verbose = 1 - elif opt in ('-q', '--quiet'): - quiet = 1 + log_level = logging.DEBUG elif opt in ('-c', '--cleanup'): cleanup = 1 elif opt in ('-t', '--test'): @@ -198,6 +222,10 @@ for opt, val in opts: httpd_service = 1 elif opt == '--httpd-no-log': httpd_no_log = 1 + elif opt == '--https': + use_ssl = 1 + elif opt == '--http2': + use_http2 = 1 elif opt == '--http-short-circuit': http_short_circuit = True elif opt == '--disable-http-v2': @@ -210,6 +238,8 @@ for opt, val in opts: fsfs_packing = 1 elif opt == '--javahl': test_javahl = 1 + elif opt == '--global-scheduler': + global_scheduler = 1 elif opt == '--swig': if val not in ['perl', 'python', 'ruby']: sys.stderr.write('Running \'%s\' swig tests not supported (yet).\n' @@ -225,7 +255,7 @@ for opt, val in opts: enable_sasl = 1 base_url = "svn://localhost/" elif opt == '--server-minor-version': - server_minor_version = val + server_minor_version = int(val) elif opt == '--bin': svn_bin = val elif opt in ('-p', '--parallel'): @@ -235,9 +265,20 @@ for opt, val in opts: elif opt == '--log-to-stdout': log_to_stdout = 1 elif opt == '--log-level': - log_level = val + log_level = getattr(logging, val, None) or int(val) elif opt == '--ssl-cert': ssl_cert = val + elif opt == '--exclusive-wc-locks': + exclusive_wc_locks = 1 + elif opt == '--memcached-server': + memcached_server = val + elif opt == '--skip-c-tests': + skip_c_tests = 1 + elif opt == '--dump-load-cross-check': + dump_load_cross_check = 1 + elif opt == '--memcached-dir': + memcached_dir = val + run_memcached = 1 # Calculate the source and test directory names abs_srcdir = os.path.abspath("") @@ -265,7 +306,12 @@ if run_httpd: if not httpd_port: httpd_port = random.randrange(1024, 30000) if not base_url: - base_url = 'http://localhost:' + str(httpd_port) + if use_ssl: + scheme = 'https' + else: + scheme = 'http' + + base_url = '%s://localhost:%d' % (scheme, httpd_port) if base_url: repo_loc = 'remote repository ' + base_url + '.' @@ -406,7 +452,9 @@ class Svnserve: args = [self.name] + self.args print('Starting %s %s' % (self.kind, self.name)) - self.proc = subprocess.Popen([self.path] + args[1:]) + env = os.environ.copy() + env['SVN_DBG_STACKTRACES_TO_STDERR'] = 'y' + self.proc = subprocess.Popen([self.path] + args[1:], env=env) def stop(self): if self.proc is not None: @@ -422,8 +470,9 @@ class Svnserve: class Httpd: "Run httpd for DAV tests" - def __init__(self, abs_httpd_dir, abs_objdir, abs_builddir, httpd_port, - service, no_log, httpv2, short_circuit, bulk_updates): + def __init__(self, abs_httpd_dir, abs_objdir, abs_builddir, abs_srcdir, + httpd_port, service, use_ssl, use_http2, no_log, httpv2, + short_circuit, bulk_updates): self.name = 'apache.exe' self.httpd_port = httpd_port self.httpd_dir = abs_httpd_dir @@ -461,11 +510,19 @@ class Httpd: self.dontdothat_file = os.path.join(abs_builddir, CMDLINE_TEST_SCRIPT_NATIVE_PATH, 'svn-test-work', 'dontdothat') + self.certfile = os.path.join(abs_builddir, + CMDLINE_TEST_SCRIPT_NATIVE_PATH, + 'svn-test-work', 'cert.pem') + self.certkeyfile = os.path.join(abs_builddir, + CMDLINE_TEST_SCRIPT_NATIVE_PATH, + 'svn-test-work', 'cert-key.pem') self.httpd_config = os.path.join(self.root, 'httpd.conf') self.httpd_users = os.path.join(self.root, 'users') self.httpd_mime_types = os.path.join(self.root, 'mime.types') + self.httpd_groups = os.path.join(self.root, 'groups') self.abs_builddir = abs_builddir self.abs_objdir = abs_objdir + self.abs_srcdir = abs_srcdir self.service_name = 'svn-test-httpd-' + str(httpd_port) if self.service: @@ -477,9 +534,13 @@ class Httpd: create_target_dir(self.root_dir) self._create_users_file() + self._create_groups_file() self._create_mime_types_file() self._create_dontdothat_file() + if use_ssl: + self._create_cert_files() + # Obtain version. version_vals = gen_obj._libraries['httpd'].version.split('.') self.httpd_ver = float('%s.%s' % (version_vals[0], version_vals[1])) @@ -488,9 +549,10 @@ class Httpd: fp = open(self.httpd_config, 'w') # Limit the number of threads (default = 64) - fp.write('\n') - fp.write('ThreadsPerChild 16\n') - fp.write('\n') + if not use_http2: + fp.write('\n') + fp.write('ThreadsPerChild 16\n') + fp.write('\n') # Global Environment fp.write('ServerRoot ' + self._quote(self.root) + '\n') @@ -508,6 +570,10 @@ class Httpd: fp.write('LogLevel Crit\n') # Write LoadModule for minimal system module + if use_ssl: + fp.write(self._sys_module('ssl_module', 'mod_ssl.so')) + if use_http2: + fp.write(self._sys_module('http2_module', 'mod_http2.so')) fp.write(self._sys_module('dav_module', 'mod_dav.so')) if self.httpd_ver >= 2.3: fp.write(self._sys_module('access_compat_module', 'mod_access_compat.so')) @@ -517,6 +583,8 @@ class Httpd: if self.httpd_ver >= 2.2: fp.write(self._sys_module('auth_basic_module', 'mod_auth_basic.so')) fp.write(self._sys_module('authn_file_module', 'mod_authn_file.so')) + fp.write(self._sys_module('authz_groupfile_module', 'mod_authz_groupfile.so')) + fp.write(self._sys_module('authz_host_module', 'mod_authz_host.so')) else: fp.write(self._sys_module('auth_module', 'mod_auth.so')) fp.write(self._sys_module('alias_module', 'mod_alias.so')) @@ -530,6 +598,18 @@ class Httpd: # And for mod_dontdothat fp.write(self._svn_module('dontdothat_module', 'mod_dontdothat.so')) + if use_ssl: + fp.write('SSLEngine on\n') + fp.write('SSLProtocol All -SSLv2 -SSLv3\n') + fp.write('SSLCertificateFile %s\n' % self._quote(self.certfile)) + fp.write('SSLCertificateKeyFile %s\n' % self._quote(self.certkeyfile)) + + if use_ssl and use_http2: + fp.write('Protocols h2 http/1.1\n') + elif use_http2: + fp.write('Protocols h2c http/1.1\n') + fp.write('H2Direct on\n') + # Don't handle .htaccess, symlinks, etc. fp.write('\n') fp.write('AllowOverride None\n') @@ -539,6 +619,7 @@ class Httpd: # Define two locations for repositories fp.write(self._svn_repo('repositories')) fp.write(self._svn_repo('local_tmp')) + fp.write(self._svn_authz_repo()) # And two redirects for the redirect tests fp.write('RedirectMatch permanent ^/svn-test-work/repositories/' @@ -569,6 +650,19 @@ class Httpd: 'jrandom', 'rayjandom']) os.spawnv(os.P_WAIT, htpasswd, ['htpasswd.exe', '-bp', self.httpd_users, 'jconstant', 'rayjandom']) + os.spawnv(os.P_WAIT, htpasswd, ['htpasswd.exe', '-bp', self.httpd_users, + '__dumpster__', '__loadster__']) + os.spawnv(os.P_WAIT, htpasswd, ['htpasswd.exe', '-bp', self.httpd_users, + 'JRANDOM', 'rayjandom']) + os.spawnv(os.P_WAIT, htpasswd, ['htpasswd.exe', '-bp', self.httpd_users, + 'JCONSTANT', 'rayjandom']) + + def _create_groups_file(self): + "Create groups for mod_authz_svn tests" + fp = open(self.httpd_groups, 'w') + fp.write('random: jrandom\n') + fp.write('constant: jconstant\n') + fp.close() def _create_mime_types_file(self): "Create empty mime.types file" @@ -588,6 +682,34 @@ class Httpd: fp.write('/ = deny\n') fp.close() + def _create_cert_files(self): + "Create certificate files" + # The unix build uses certificates encoded in davautocheck.sh + # Let's just read them from there + + sh_path = os.path.join(self.abs_srcdir, 'subversion', 'tests', 'cmdline', + 'davautocheck.sh') + sh = open(sh_path).readlines() + + def cert_extract(lines, what): + r = [] + pattern = r'cat\s*\>\s*' + re.escape(what) + r'\s*\<\<([A-Z_a-z0-9]+)' + exit_marker = None + for i in lines: + if exit_marker: + if i.startswith(exit_marker): + return r + r.append(i) + else: + m = re.match(pattern, i) + if m: + exit_marker = m.groups(1) + + cert_file = cert_extract(sh, '"$SSL_CERTIFICATE_FILE"') + cert_key = cert_extract(sh, '"$SSL_CERTIFICATE_KEY_FILE"') + open(self.certfile, 'w').write(''.join(cert_file)) + open(self.certkeyfile, 'w').write(''.join(cert_key)) + def _sys_module(self, name, path): full_path = os.path.join(self.httpd_dir, 'modules', path) return 'LoadModule ' + name + " " + self._quote(full_path) + '\n' @@ -629,12 +751,163 @@ class Httpd: ' DontDoThatConfigFile ' + self._quote(self.dontdothat_file) + '\n' \ '\n' + def _svn_authz_repo(self): + local_tmp = os.path.join(self.abs_builddir, + CMDLINE_TEST_SCRIPT_NATIVE_PATH, + 'svn-test-work', 'local_tmp') + return \ + '' + '\n' \ + ' DAV svn' + '\n' \ + ' SVNParentPath ' + local_tmp + '\n' \ + ' AuthzSVNAccessFile ' + self._quote(self.authz_file) + '\n' \ + ' SVNAdvertiseV2Protocol ' + self.httpv2_option + '\n' \ + ' SVNListParentPath On' + '\n' \ + ' ' + '\n' \ + ' Require all granted' + '\n' \ + ' ' + '\n' \ + ' ' + '\n' \ + ' Allow from all' + '\n' \ + ' ' + '\n' \ + ' SVNPathAuthz ' + self.path_authz_option + '\n' \ + '' + '\n' \ + '' + '\n' \ + ' DAV svn' + '\n' \ + ' SVNParentPath ' + local_tmp + '\n' \ + ' AuthzSVNAccessFile ' + self._quote(self.authz_file) + '\n' \ + ' SVNAdvertiseV2Protocol ' + self.httpv2_option + '\n' \ + ' SVNListParentPath On' + '\n' \ + ' AuthType Basic' + '\n' \ + ' AuthName "Subversion Repository"' + '\n' \ + ' AuthUserFile ' + self._quote(self.httpd_users) + '\n' \ + ' Require valid-user' + '\n' \ + ' Satisfy Any' + '\n' \ + ' SVNPathAuthz ' + self.path_authz_option + '\n' \ + '' + '\n' \ + '' + '\n' \ + ' DAV svn' + '\n' \ + ' SVNParentPath ' + local_tmp + '\n' \ + ' AuthzSVNAccessFile ' + self._quote(self.authz_file) + '\n' \ + ' SVNAdvertiseV2Protocol ' + self.httpv2_option + '\n' \ + ' SVNListParentPath On' + '\n' \ + ' AuthType Basic' + '\n' \ + ' AuthName "Subversion Repository"' + '\n' \ + ' AuthUserFile ' + self._quote(self.httpd_users) + '\n' \ + ' Require valid-user' + '\n' \ + ' AuthzSVNNoAuthWhenAnonymousAllowed On' + '\n' \ + ' SVNPathAuthz On' + '\n' \ + '' + '\n' \ + '' + '\n' \ + ' DAV svn' + '\n' \ + ' SVNParentPath ' + local_tmp + '\n' \ + ' AuthzSVNAccessFile ' + self._quote(self.authz_file) + '\n' \ + ' SVNAdvertiseV2Protocol ' + self.httpv2_option + '\n' \ + ' SVNListParentPath On' + '\n' \ + ' AuthType Basic' + '\n' \ + ' AuthName "Subversion Repository"' + '\n' \ + ' AuthUserFile ' + self._quote(self.httpd_users) + '\n' \ + ' Require valid-user' + '\n' \ + ' SVNPathAuthz ' + self.path_authz_option + '\n' \ + '' + '\n' \ + '' + '\n' \ + ' DAV svn' + '\n' \ + ' SVNParentPath ' + local_tmp + '\n' \ + ' AuthzSVNAccessFile ' + self._quote(self.authz_file) + '\n' \ + ' SVNAdvertiseV2Protocol ' + self.httpv2_option + '\n' \ + ' SVNListParentPath On' + '\n' \ + ' AuthType Basic' + '\n' \ + ' AuthName "Subversion Repository"' + '\n' \ + ' AuthUserFile ' + self._quote(self.httpd_users) + '\n' \ + ' Require valid-user' + '\n' \ + ' AuthzSVNAnonymous Off' + '\n' \ + ' SVNPathAuthz On' + '\n' \ + '' + '\n' \ + '' + '\n' \ + ' DAV svn' + '\n' \ + ' SVNParentPath ' + local_tmp + '\n' \ + ' AuthzSVNAccessFile ' + self._quote(self.authz_file) + '\n' \ + ' SVNAdvertiseV2Protocol ' + self.httpv2_option + '\n' \ + ' SVNListParentPath On' + '\n' \ + ' AuthType Basic' + '\n' \ + ' AuthName "Subversion Repository"' + '\n' \ + ' AuthUserFile ' + self._quote(self.httpd_users) + '\n' \ + ' Require valid-user' + '\n' \ + ' AuthzForceUsernameCase Lower' + '\n' \ + ' SVNPathAuthz ' + self.path_authz_option + '\n' \ + '' + '\n' \ + '' + '\n' \ + ' DAV svn' + '\n' \ + ' SVNParentPath ' + local_tmp + '\n' \ + ' AuthzSVNAccessFile ' + self._quote(self.authz_file) + '\n' \ + ' SVNAdvertiseV2Protocol ' + self.httpv2_option + '\n' \ + ' SVNListParentPath On' + '\n' \ + ' AuthType Basic' + '\n' \ + ' AuthName "Subversion Repository"' + '\n' \ + ' AuthUserFile ' + self._quote(self.httpd_users) + '\n' \ + ' Require valid-user' + '\n' \ + ' AuthzForceUsernameCase Lower' + '\n' \ + ' SVNPathAuthz ' + self.path_authz_option + '\n' \ + '' + '\n' \ + '' + '\n' \ + ' DAV svn' + '\n' \ + ' SVNParentPath ' + local_tmp + '\n' \ + ' AuthzSVNAccessFile ' + self._quote(self.authz_file) + '\n' \ + ' SVNAdvertiseV2Protocol ' + self.httpv2_option + '\n' \ + ' SVNListParentPath On' + '\n' \ + ' AuthType Basic' + '\n' \ + ' AuthName "Subversion Repository"' + '\n' \ + ' AuthUserFile ' + self._quote(self.httpd_users) + '\n' \ + ' AuthGroupFile ' + self._quote(self.httpd_groups) + '\n' \ + ' Require group random' + '\n' \ + ' AuthzSVNAuthoritative Off' + '\n' \ + ' SVNPathAuthz On' + '\n' \ + '' + '\n' \ + '' + '\n' \ + '' + '\n' \ + ' DAV svn' + '\n' \ + ' SVNParentPath ' + local_tmp + '\n' \ + ' AuthzSVNAccessFile ' + self._quote(self.authz_file) + '\n' \ + ' SVNAdvertiseV2Protocol ' + self.httpv2_option + '\n' \ + ' SVNListParentPath On' + '\n' \ + ' AuthType Basic' + '\n' \ + ' AuthName "Subversion Repository"' + '\n' \ + ' AuthUserFile ' + self._quote(self.httpd_users) + '\n' \ + ' AuthzSendForbiddenOnFailure On' + '\n' \ + ' Satisfy All' + '\n' \ + ' ' + '\n' \ + ' Require valid-user' + '\n' \ + ' Require expr req(\'ALLOW\') == \'1\'' + '\n' \ + ' ' + '\n' \ + ' SVNPathAuthz ' + self.path_authz_option + '\n' \ + '' + '\n' \ + ''+ '\n' \ + ' DAV svn' + '\n' \ + ' SVNParentPath ' + local_tmp + '\n' \ + ' AuthzSVNAccessFile ' + self._quote(self.authz_file) + '\n' \ + ' SVNAdvertiseV2Protocol ' + self.httpv2_option + '\n' \ + ' SVNListParentPath On' + '\n' \ + ' AuthType Basic' + '\n' \ + ' AuthName "Subversion Repository"' + '\n' \ + ' AuthUserFile ' + self._quote(self.httpd_users) + '\n' \ + ' AuthzSendForbiddenOnFailure On' + '\n' \ + ' Satisfy All' + '\n' \ + ' ' + '\n' \ + ' Require valid-user' + '\n' \ + ' Require expr req(\'ALLOW\') == \'1\'' + '\n' \ + ' ' + '\n' \ + ' SVNPathAuthz ' + self.path_authz_option + '\n' \ + '' + '\n' \ + '' + '\n' \ + def start(self): if self.service: self._start_service() else: self._start_daemon() + # Avoid output from starting and preparing between test results + sys.stderr.flush() + sys.stdout.flush() + def stop(self): if self.service: self._stop_service() @@ -672,6 +945,45 @@ class Httpd: pass print('Httpd.stop_daemon not implemented') +class Memcached: + "Run memcached for tests" + def __init__(self, abs_memcached_dir, memcached_server): + self.name = 'memcached.exe' + + self.memcached_host, self.memcached_port = memcached_server.split(':') + self.memcached_dir = abs_memcached_dir + + self.proc = None + self.path = os.path.join(self.memcached_dir, self.name) + + self.memcached_args = [ + self.name, + '-p', self.memcached_port, + '-l', self.memcached_host + ] + + def __del__(self): + "Stop memcached when the object is deleted" + self.stop() + + def start(self): + "Start memcached as daemon" + print('Starting %s as daemon' % self.name) + print(self.memcached_args) + self.proc = subprocess.Popen([self.path] + self.memcached_args) + + def stop(self): + "Stop memcached" + if self.proc is not None: + try: + print('Stopping %s' % self.name) + self.proc.poll(); + if self.proc.returncode is None: + self.proc.kill(); + return + except AttributeError: + pass + # Move the binaries to the test directory create_target_dir(abs_builddir) locate_libs() @@ -692,19 +1004,27 @@ create_target_dir(CMDLINE_TEST_SCRIPT_NA abs_builddir = fix_case(abs_builddir) daemon = None +memcached = None # Run the tests # No need to start any servers if we are only listing the tests. if not list_tests: + if run_memcached: + memcached = Memcached(memcached_dir, memcached_server) + memcached.start() + if run_svnserve: daemon = Svnserve(svnserve_args, objdir, abs_objdir, abs_builddir) if run_httpd: - daemon = Httpd(abs_httpd_dir, abs_objdir, abs_builddir, httpd_port, - httpd_service, httpd_no_log, - advertise_httpv2, http_short_circuit, + daemon = Httpd(abs_httpd_dir, abs_objdir, abs_builddir, abs_srcdir, + httpd_port, httpd_service, use_ssl, use_http2, + httpd_no_log, advertise_httpv2, http_short_circuit, http_bulk_updates) + if use_ssl and not ssl_cert: + ssl_cert = daemon.certfile + # Start service daemon, if any if daemon: daemon.start() @@ -750,16 +1070,36 @@ if not test_javahl and not test_swig: log_file = os.path.join(abs_builddir, log) fail_log_file = os.path.join(abs_builddir, faillog) + if run_httpd: + httpd_version = gen_obj._libraries['httpd'].version + else: + httpd_version = None + + opts, args = run_tests.create_parser().parse_args([]) + opts.url = base_url + opts.fs_type = fs_type + opts.global_scheduler = global_scheduler + opts.http_library = 'serf' + opts.server_minor_version = server_minor_version + opts.cleanup = cleanup + opts.enable_sasl = enable_sasl + opts.parallel = parallel + opts.config_file = config_file + opts.fsfs_sharding = fsfs_sharding + opts.fsfs_packing = fsfs_packing + opts.list_tests = list_tests + opts.svn_bin = svn_bin + opts.mode_filter = mode_filter + opts.milestone_filter = milestone_filter + opts.httpd_version = httpd_version + opts.set_log_level = log_level + opts.ssl_cert = ssl_cert + opts.exclusive_wc_locks = exclusive_wc_locks + opts.memcached_server = memcached_server + opts.skip_c_tests = skip_c_tests + opts.dump_load_cross_check = dump_load_cross_check th = run_tests.TestHarness(abs_srcdir, abs_builddir, - log_file, - fail_log_file, - base_url, fs_type, 'serf', - server_minor_version, not quiet, - cleanup, enable_sasl, parallel, config_file, - fsfs_sharding, fsfs_packing, - list_tests, svn_bin, mode_filter, - milestone_filter, - set_log_level=log_level, ssl_cert=ssl_cert) + log_file, fail_log_file, opts) old_cwd = os.getcwd() try: os.chdir(abs_builddir) @@ -792,6 +1132,9 @@ elif test_javahl: if (objdir == 'Debug'): args = args + ('-Xcheck:jni',) + if cleanup: + args = args + ('-Dtest.cleanup=1',) + args = args + ( '-Dtest.rootdir=' + os.path.join(abs_builddir, 'javahl'), '-Dtest.srcdir=' + os.path.join(abs_srcdir, @@ -976,6 +1319,9 @@ elif test_swig == 'ruby': if daemon: del daemon +if memcached: + del memcached + # Remove the execs again for tgt in copied_execs: try: