Return-Path: Delivered-To: apmail-ant-notifications-archive@minotaur.apache.org Received: (qmail 82321 invoked from network); 10 Jul 2009 17:37:21 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 10 Jul 2009 17:37:21 -0000 Received: (qmail 88652 invoked by uid 500); 10 Jul 2009 17:37:31 -0000 Delivered-To: apmail-ant-notifications-archive@ant.apache.org Received: (qmail 88592 invoked by uid 500); 10 Jul 2009 17:37:31 -0000 Mailing-List: contact notifications-help@ant.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ant.apache.org Delivered-To: mailing list notifications@ant.apache.org Received: (qmail 88583 invoked by uid 99); 10 Jul 2009 17:37:31 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 Jul 2009 17:37:31 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 Jul 2009 17:37:27 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id C297B23888C5; Fri, 10 Jul 2009 17:37:05 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r793048 - in /ant/core/trunk: ./ docs/webtest/gettest/ src/main/org/apache/tools/ant/taskdefs/ src/tests/antunit/taskdefs/ Date: Fri, 10 Jul 2009 17:37:05 -0000 To: notifications@ant.apache.org From: jkf@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20090710173705.C297B23888C5@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: jkf Date: Fri Jul 10 17:37:04 2009 New Revision: 793048 URL: http://svn.apache.org/viewvc?rev=793048&view=rev Log: Also allow redirect from http to https a subset of what was requested in pr 47433. Tests from https to http v.v. have also been executed I couldn't find a https space for public testcases though. Added: ant/core/trunk/docs/webtest/gettest/ ant/core/trunk/docs/webtest/gettest/.htaccess ant/core/trunk/docs/webtest/gettest/testother.txt ant/core/trunk/docs/webtest/gettest/testperm.txt ant/core/trunk/docs/webtest/gettest/testredir5.txt ant/core/trunk/docs/webtest/gettest/testtemp.txt ant/core/trunk/src/tests/antunit/taskdefs/get-test.xml Modified: ant/core/trunk/WHATSNEW ant/core/trunk/src/main/org/apache/tools/ant/taskdefs/Get.java Modified: ant/core/trunk/WHATSNEW URL: http://svn.apache.org/viewvc/ant/core/trunk/WHATSNEW?rev=793048&r1=793047&r2=793048&view=diff ============================================================================== --- ant/core/trunk/WHATSNEW (original) +++ ant/core/trunk/WHATSNEW Fri Jul 10 17:37:04 2009 @@ -394,6 +394,9 @@ Other changes: -------------- + * The get task now also follows redirects from http to https + Bugzilla report 47433 + * A HostInfo task was added performing information on hosts, including info on the host ant is running on. Bugzilla reports 45861 and 31164. Added: ant/core/trunk/docs/webtest/gettest/.htaccess URL: http://svn.apache.org/viewvc/ant/core/trunk/docs/webtest/gettest/.htaccess?rev=793048&view=auto ============================================================================== --- ant/core/trunk/docs/webtest/gettest/.htaccess (added) +++ ant/core/trunk/docs/webtest/gettest/.htaccess Fri Jul 10 17:37:04 2009 @@ -0,0 +1,10 @@ +Redirect seeother /webtest/gettest/other.txt /webtest/gettest/testother.txt +Redirect temp /webtest/gettest/temp.txt /webtest/gettest/testtemp.txt +Redirect permanent /webtest/gettest/permanent.txt /webtest/gettest/testperm.txt +Redirect /webtest/gettest/infinite.txt /webtest/gettest/infinite2.txt +Redirect /webtest/gettest/infinite2.txt /webtest/gettest/infinite.txt +Redirect /webtest/gettest/redir5.txt /webtest/gettest/redir5-1.txt +Redirect /webtest/gettest/redir5-1.txt /webtest/gettest/redir5-2.txt +Redirect /webtest/gettest/redir5-2.txt /webtest/gettest/redir5-3.txt +Redirect /webtest/gettest/redir5-3.txt /webtest/gettest/redir5-4.txt +Redirect /webtest/gettest/redir5-4.txt /webtest/gettest/testredir5.txt Added: ant/core/trunk/docs/webtest/gettest/testother.txt URL: http://svn.apache.org/viewvc/ant/core/trunk/docs/webtest/gettest/testother.txt?rev=793048&view=auto ============================================================================== --- ant/core/trunk/docs/webtest/gettest/testother.txt (added) +++ ant/core/trunk/docs/webtest/gettest/testother.txt Fri Jul 10 17:37:04 2009 @@ -0,0 +1 @@ +seeother redirect succeeded Added: ant/core/trunk/docs/webtest/gettest/testperm.txt URL: http://svn.apache.org/viewvc/ant/core/trunk/docs/webtest/gettest/testperm.txt?rev=793048&view=auto ============================================================================== --- ant/core/trunk/docs/webtest/gettest/testperm.txt (added) +++ ant/core/trunk/docs/webtest/gettest/testperm.txt Fri Jul 10 17:37:04 2009 @@ -0,0 +1 @@ +permanent redirect succeeded Added: ant/core/trunk/docs/webtest/gettest/testredir5.txt URL: http://svn.apache.org/viewvc/ant/core/trunk/docs/webtest/gettest/testredir5.txt?rev=793048&view=auto ============================================================================== --- ant/core/trunk/docs/webtest/gettest/testredir5.txt (added) +++ ant/core/trunk/docs/webtest/gettest/testredir5.txt Fri Jul 10 17:37:04 2009 @@ -0,0 +1 @@ +5 levels of redirect succeeded Added: ant/core/trunk/docs/webtest/gettest/testtemp.txt URL: http://svn.apache.org/viewvc/ant/core/trunk/docs/webtest/gettest/testtemp.txt?rev=793048&view=auto ============================================================================== --- ant/core/trunk/docs/webtest/gettest/testtemp.txt (added) +++ ant/core/trunk/docs/webtest/gettest/testtemp.txt Fri Jul 10 17:37:04 2009 @@ -0,0 +1 @@ +temporary redirect succeeded Modified: ant/core/trunk/src/main/org/apache/tools/ant/taskdefs/Get.java URL: http://svn.apache.org/viewvc/ant/core/trunk/src/main/org/apache/tools/ant/taskdefs/Get.java?rev=793048&r1=793047&r2=793048&view=diff ============================================================================== --- ant/core/trunk/src/main/org/apache/tools/ant/taskdefs/Get.java (original) +++ ant/core/trunk/src/main/org/apache/tools/ant/taskdefs/Get.java Fri Jul 10 17:37:04 2009 @@ -18,12 +18,8 @@ package org.apache.tools.ant.taskdefs; -import org.apache.tools.ant.BuildException; -import org.apache.tools.ant.Project; -import org.apache.tools.ant.Task; -import org.apache.tools.ant.util.FileUtils; - import java.io.File; +import java.io.FileNotFoundException; import java.io.FileOutputStream; import java.io.IOException; import java.io.InputStream; @@ -33,6 +29,13 @@ import java.net.URL; import java.net.URLConnection; import java.util.Date; +import java.util.HashSet; +import java.util.Set; + +import org.apache.tools.ant.BuildException; +import org.apache.tools.ant.Project; +import org.apache.tools.ant.Task; +import org.apache.tools.ant.util.FileUtils; /** * Gets a particular file from a URL source. @@ -49,6 +52,10 @@ private static final int DOTS_PER_LINE = 50; private static final int BIG_BUFFER_SIZE = 100 * 1024; private static final FileUtils FILE_UTILS = FileUtils.getFileUtils(); + private static final int REDIRECT_LIMIT = 25; + + private static final String HTTP = "http"; + private static final String HTTPS = "https"; private URL source; // required private File dest; // required @@ -361,6 +368,7 @@ } private class GetThread extends Thread { + private final boolean hasTimestamp; private final long timestamp; private final DownloadProgress progress; @@ -371,6 +379,8 @@ private BuildException exception = null; private InputStream is = null; private OutputStream os = null; + private URLConnection connection; + private int redirections = 0; GetThread(boolean h, long t, DownloadProgress p, int l) { hasTimestamp = h; @@ -390,10 +400,64 @@ } private boolean get() throws IOException, BuildException { - //set up the URL connection - URLConnection connection = source.openConnection(); - //modify the headers - //NB: things like user authentication could go in here too. + + connection = openConnection(source); + + if (connection == null) + { + return false; + } + + boolean downloadSucceeded = downloadFile(); + + //if (and only if) the use file time option is set, then + //the saved file now has its timestamp set to that of the + //downloaded file + if (downloadSucceeded && useTimestamp) { + updateTimeStamp(); + } + + return downloadSucceeded; + } + + + private boolean redirectionAllowed(URL aSource, URL aDest) { + if (!(aSource.getProtocol().equals(aDest.getProtocol()) || (HTTP + .equals(aSource.getProtocol()) && HTTPS.equals(aDest + .getProtocol())))) { + String message = "Redirection detected from " + + aSource.getProtocol() + " to " + aDest.getProtocol() + + ". Protocol switch unsafe, not allowed."; + if (ignoreErrors) { + log(message, logLevel); + return false; + } else { + throw new BuildException(message); + } + } + + redirections++; + if (redirections > REDIRECT_LIMIT) { + String message = "More than " + REDIRECT_LIMIT + + " times redirected, giving up"; + if (ignoreErrors) { + log(message, logLevel); + return false; + } else { + throw new BuildException(message); + } + } + + + return true; + } + + private URLConnection openConnection(URL aSource) throws IOException { + + // set up the URL connection + URLConnection connection = aSource.openConnection(); + // modify the headers + // NB: things like user authentication could go in here too. if (hasTimestamp) { connection.setIfModifiedSince(timestamp); } @@ -401,45 +465,65 @@ if (uname != null || pword != null) { String up = uname + ":" + pword; String encoding; - //we do not use the sun impl for portability, - //and always use our own implementation for consistent - //testing + // we do not use the sun impl for portability, + // and always use our own implementation for consistent + // testing Base64Converter encoder = new Base64Converter(); encoding = encoder.encode(up.getBytes()); - connection.setRequestProperty ("Authorization", - "Basic " + encoding); + connection.setRequestProperty("Authorization", "Basic " + + encoding); } - //connect to the remote site (may take some time) + if (connection instanceof HttpURLConnection) { + ((HttpURLConnection) connection) + .setInstanceFollowRedirects(false); + } + // connect to the remote site (may take some time) connection.connect(); - //next test for a 304 result (HTTP only) + + // First check on a 301 / 302 (moved) response (HTTP only) if (connection instanceof HttpURLConnection) { - HttpURLConnection httpConnection - = (HttpURLConnection) connection; + HttpURLConnection httpConnection = (HttpURLConnection) connection; + // httpConnection.setInstanceFollowRedirects(false); + // httpConnection.setUseCaches(false); + int responseCode = httpConnection.getResponseCode(); + if (responseCode == HttpURLConnection.HTTP_MOVED_PERM || + responseCode == HttpURLConnection.HTTP_MOVED_TEMP || + responseCode == HttpURLConnection.HTTP_SEE_OTHER) + { + String newLocation = httpConnection.getHeaderField("Location"); + String message = aSource + + (responseCode == HttpURLConnection.HTTP_MOVED_PERM ? " permanently" + : "") + " moved to " + newLocation; + log(message, logLevel); + URL newURL = new URL(newLocation); + if (!redirectionAllowed(aSource, newURL)) + { + return null; + } + return openConnection(newURL); + } + // next test for a 304 result (HTTP only) long lastModified = httpConnection.getLastModified(); - if (httpConnection.getResponseCode() - == HttpURLConnection.HTTP_NOT_MODIFIED - || (lastModified != 0 && hasTimestamp - && timestamp >= lastModified)) { - //not modified so no file download. just return - //instead and trace out something so the user - //doesn't think that the download happened when it - //didn't + if (responseCode == HttpURLConnection.HTTP_NOT_MODIFIED + || (lastModified != 0 && hasTimestamp && timestamp >= lastModified)) { + // not modified so no file download. just return + // instead and trace out something so the user + // doesn't think that the download happened when it + // didn't log("Not modified - so not downloaded", logLevel); - return false; + return null; } // test for 401 result (HTTP only) - if (httpConnection.getResponseCode() - == HttpURLConnection.HTTP_UNAUTHORIZED) { + if (responseCode == HttpURLConnection.HTTP_UNAUTHORIZED) { String message = "HTTP Authorization failure"; if (ignoreErrors) { log(message, logLevel); - return false; + return null; } else { throw new BuildException(message); } } - } //REVISIT: at this point even non HTTP connections may @@ -447,11 +531,15 @@ //the date of the content and skip the write if it is not //newer. Some protocols (FTP) don't include dates, of //course. + return connection; + } + private boolean downloadFile() + throws FileNotFoundException, IOException { for (int i = 0; i < NUMBER_RETRIES; i++) { - //this three attempt trick is to get round quirks in different - //Java implementations. Some of them take a few goes to bind - //property; we ignore the first couple of such failures. + // this three attempt trick is to get round quirks in different + // Java implementations. Some of them take a few goes to bind + // property; we ignore the first couple of such failures. try { is = connection.getInputStream(); break; @@ -464,8 +552,8 @@ if (ignoreErrors) { return false; } - throw new BuildException("Can't get " + source + " to " - + dest, getLocation()); + throw new BuildException("Can't get " + source + " to " + dest, + getLocation()); } os = new FileOutputStream(dest); @@ -491,26 +579,23 @@ } } progress.endDownload(); - - //if (and only if) the use file time option is set, then - //the saved file now has its timestamp set to that of the - //downloaded file - if (useTimestamp) { - long remoteTimestamp = connection.getLastModified(); - if (verbose) { - Date t = new Date(remoteTimestamp); - log("last modified = " + t.toString() - + ((remoteTimestamp == 0) - ? " - using current time instead" - : ""), logLevel); - } - if (remoteTimestamp != 0) { - FILE_UTILS.setFileLastModified(dest, remoteTimestamp); - } - } return true; } + private void updateTimeStamp() { + long remoteTimestamp = connection.getLastModified(); + if (verbose) { + Date t = new Date(remoteTimestamp); + log("last modified = " + t.toString() + + ((remoteTimestamp == 0) + ? " - using current time instead" + : ""), logLevel); + } + if (remoteTimestamp != 0) { + FILE_UTILS.setFileLastModified(dest, remoteTimestamp); + } + } + /** * Has the download completed successfully? * Added: ant/core/trunk/src/tests/antunit/taskdefs/get-test.xml URL: http://svn.apache.org/viewvc/ant/core/trunk/src/tests/antunit/taskdefs/get-test.xml?rev=793048&view=auto ============================================================================== --- ant/core/trunk/src/tests/antunit/taskdefs/get-test.xml (added) +++ ant/core/trunk/src/tests/antunit/taskdefs/get-test.xml Fri Jul 10 17:37:04 2009 @@ -0,0 +1,90 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +