commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From d...@multitask.com.au
Subject Re: [HttpClient] Retriving an unbuffered stream
Date Sun, 01 Sep 2002 02:33:22 GMT
Hi Ortwin,

where did we get to with the unbuffered stream discussion. I went back and 
re-read the posts and it seemed that the patch was acceptable....

Did I miss something, or is it waiting on a committer?
--
dIon Gillard, Multitask Consulting
Work:      http://www.multitask.com.au
Developers: http://adslgateway.multitask.com.au/developers


"Ortwin Glück" <ortwin.glueck@nose.ch> wrote on 28/08/2002 08:35:05 PM:

> Finally, the footer stuff made it into the code. There is a small test 
> class as well.
> 
> Patch file is for the src directory.
> 
> Odi
> /*
>  * $Header$
>  * $Revision$
>  * $Date$
>  * ====================================================================
>  *
>  * The Apache Software License, Version 1.1
>  *
>  * Copyright (c) 1999-2002 The Apache Software Foundation.  All rights
>  * reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>  * are met:
>  *
>  * 1. Redistributions of source code must retain the above copyright
>  *    notice, this list of conditions and the following disclaimer.
>  *
>  * 2. Redistributions in binary form must reproduce the above copyright
>  *    notice, this list of conditions and the following disclaimer in
>  *    the documentation and/or other materials provided with the
>  *    distribution.
>  *
>  * 3. The end-user documentation included with the redistribution, if
>  *    any, must include the following acknowlegement:
>  *       "This product includes software developed by the
>  *        Apache Software Foundation (http://www.apache.org/)."
>  *    Alternately, this acknowlegement may appear in the software 
itself,
>  *    if and wherever such third-party acknowlegements normally appear.
>  *
>  * 4. The names "The Jakarta Project", "HttpClient", and "Apache 
Software
>  *    Foundation" must not be used to endorse or promote products 
derived
>  *    from this software without prior written permission. For written
>  *    permission, please contact apache@apache.org.
>  *
>  * 5. Products derived from this software may not be called "Apache"
>  *    nor may "Apache" appear in their names without prior written
>  *    permission of the Apache Group.
>  *
>  * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED
>  * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
>  * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
>  * DISCLAIMED.  IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION OR
>  * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
>  * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
>  * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
>  * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
>  * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
>  * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
>  * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
>  * SUCH DAMAGE.
>  * ====================================================================
>  *
>  * This software consists of voluntary contributions made by many
>  * individuals on behalf of the Apache Software Foundation.  For more
>  * information on the Apache Software Foundation, please see
>  * <http://www.apache.org/>.
>  *
>  * [Additional notices, if required by prior licensing conditions]
>  *
>  */
> 
> package org.apache.commons.httpclient;
> 
> import java.io.*;
> 
> /**
>  * Closes a HttpConnection as soon as the end of the stream is reached.
>  * @author Ortwin Glück
>  *
>  * @since 2.0
>  */
> 
> class AutoCloseInputStream extends FilterInputStream {
>     private HttpConnection conn;
> 
>     public AutoCloseInputStream(InputStream in, HttpConnection conn) {
>         super(in);
>         this.conn = conn;
>     }
> 
>     public int read() throws java.io.IOException {
>         int l = super.read();
>         if (l == -1) conn.close();
>         return l;
>     }
> 
>     public int read(byte[] b, int off, int len) throws 
java.io.IOException {
>         int l = super.read(b,  off,  len);
>         if (l == -1) conn.close();
>         return l;
>     }
> 
>     public int read(byte[] b) throws java.io.IOException {
>         int l = super.read(b);
>         if (l == -1) conn.close();
>         return l;
>     }
> }/*
>  * $Header$
>  * $Revision$
>  * $Date$
>  * ====================================================================
>  *
>  * The Apache Software License, Version 1.1
>  *
>  * Copyright (c) 1999-2002 The Apache Software Foundation.  All rights
>  * reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>  * are met:
>  *
>  * 1. Redistributions of source code must retain the above copyright
>  *    notice, this list of conditions and the following disclaimer.
>  *
>  * 2. Redistributions in binary form must reproduce the above copyright
>  *    notice, this list of conditions and the following disclaimer in
>  *    the documentation and/or other materials provided with the
>  *    distribution.
>  *
>  * 3. The end-user documentation included with the redistribution, if
>  *    any, must include the following acknowlegement:
>  *       "This product includes software developed by the
>  *        Apache Software Foundation (http://www.apache.org/)."
>  *    Alternately, this acknowlegement may appear in the software 
itself,
>  *    if and wherever such third-party acknowlegements normally appear.
>  *
>  * 4. The names "The Jakarta Project", "HttpClient", and "Apache 
Software
>  *    Foundation" must not be used to endorse or promote products 
derived
>  *    from this software without prior written permission. For written
>  *    permission, please contact apache@apache.org.
>  *
>  * 5. Products derived from this software may not be called "Apache"
>  *    nor may "Apache" appear in their names without prior written
>  *    permission of the Apache Group.
>  *
>  * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED
>  * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
>  * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
>  * DISCLAIMED.  IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION OR
>  * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
>  * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
>  * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
>  * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
>  * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
>  * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
>  * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
>  * SUCH DAMAGE.
>  * ====================================================================
>  *
>  * This software consists of voluntary contributions made by many
>  * individuals on behalf of the Apache Software Foundation.  For more
>  * information on the Apache Software Foundation, please see
>  * <http://www.apache.org/>.
>  *
>  * [Additional notices, if required by prior licensing conditions]
>  *
>  */
> 
> package org.apache.commons.httpclient;
> 
> import java.io.*;
> import java.util.*;
> 
> /**
>  * <p>Transparently coalesces chunks of a HTTP stream that uses 
> Transfer-Encoding
>  * chunked.</p>
>  * @author Ortwin Glück
>  * @since 2.0
>  */
> 
> public class ChunkedInputStream extends InputStream {
>     private InputStream in;
>     private int chunkSize, pos;
>     private boolean eof = false;
>     private static final String HTTP_ENC = "US-ASCII";
>     private HttpMethod method;
> 
>     public ChunkedInputStream(InputStream in, HttpMethod method) 
> throws IOException {
>         this.in = in;
>         this.method = method;
>         this.chunkSize = getChunkSize();
>         this.pos = 0;
>     }
> 
>     /**
>      * Returns all the data in a chunked stream in coalesced form. A 
chunk is
>      * followed by a CRLF. The method returns -1 as soon as a 
> chunksize of 0 is
>      * detected.
>      * Footers are read automcatically at the end of the stream and 
> can be obtained
>      * with the getFooters() method.
>      *
>      * @return -1 of the end of the stream has been reached or the 
> next data byte
>      * @throws IOException
>      */
>     public int read() throws IOException {
>         if (eof) return -1;
>         if (pos >= chunkSize) {
>             nextChunk();
>             if (eof) return -1;
>         }
>         pos++;
>         return in.read();
>     }
> 
>     public int read(byte[] b, int off, int len) throws 
java.io.IOException {
>         if (eof) return -1;
>         if (pos >= chunkSize) {
>             nextChunk();
>             if (eof) return -1;
>         }
>         len = Math.min(len, chunkSize);
>         int count = in.read(b, off, len);
>         pos += count;
>         return count;
>     }
> 
>     public int read(byte[] b) throws java.io.IOException {
>         return read(b, 0, b.length);
>     }
> 
>     private void nextChunk() throws IOException {
>         int cr = in.read();
>         int lf = in.read();
>         if ((cr != '\r') &&
>             (lf != '\n')) throw new IOException("CRLF expected at 
> end of chunk: "+cr+"/"+lf);
>         chunkSize = getChunkSize();
>         pos = 0;
>         if (chunkSize == 0) {
>             eof = true;
>             parseFooters();
>         }
>     }
> 
>     /**
>      * Expects the stream to start with a chunksize in hex with 
> optional comments
>      * after a semicolon. The line must end with a CRLF:
>      * "a3; some comment\r\n"
>      * Positions the stream at the start of the next line.
>      *
>      * @return the chunk size as integer
>      * @throws IOException when the chunk size could not be parsed
>      */
>     private int getChunkSize() throws IOException {
>         ByteArrayOutputStream baos = new ByteArrayOutputStream();
>         int b = in.read();
>         int state = 0;
>         while (state != 2) {
>             if (b == -1) throw new IOException("chunked stream ended
> unexpectedly");
>             switch (state) {
>                 case 0:
>                     if (b == '\r') state = 1;
>                     break;
>                 case 1:
>                     if (b == '\n') state = 2;
>                     else state = 0;
>                     break;
>                 default: throw new RuntimeException("assertion failed");
>             }
>             if (state == 0) baos.write(b);
>             if (state != 2) b = in.read();
>         }
> 
>         //parse data
>         String dataString = new String(baos.toByteArray(), HTTP_ENC);
>         int separator = dataString.indexOf(';');
>         if (separator > 0) dataString = dataString.substring(0, 
> separator).trim();
> 
>         int result;
>         try {
>             result = Integer.parseInt(dataString, 16);
>         } catch(NumberFormatException e) {
>             throw new IOException("Bad chunk size: "+dataString);
>         }
>         return result;
>     }
> 
>     /**
>      * Stores the footers into map of Headers
>      */
>     private void parseFooters() throws IOException {
>         String line = readLine();
>         while ((line != null) && (!line.equals(""))) {
>             int colonPos = line.indexOf(':');
>             if (colonPos != -1) {
>                 String key = line.substring(0, colonPos).trim();
>                 String val = line.substring(colonPos+1).trim();
>                 Header footer = new Header(key, val);
>                 method.addResponseFooter(footer);
>             }
>             line = readLine();
>         }
>     }
> 
>     private String readLine() throws IOException {
>         StringBuffer buf = new StringBuffer();
>         for(;;) {
>             int ch = in.read();
>             if(ch < 0) {
>                 if(buf.length() == 0) {
>                     return null;
>                 } else {
>                     break;
>                 }
>             } else if (ch == '\r') {
>                 continue;
>             } else if (ch == '\n') {
>                 break;
>             }
>             buf.append((char)ch);
>         }
>         return (buf.toString());
>     }
> 
>     public void close() throws IOException {
>         in.close();
>     }
> }/*
>  * $Header$
>  * $Revision$
>  * $Date$
>  * ====================================================================
>  *
>  * The Apache Software License, Version 1.1
>  *
>  * Copyright (c) 1999-2002 The Apache Software Foundation.  All rights
>  * reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>  * are met:
>  *
>  * 1. Redistributions of source code must retain the above copyright
>  *    notice, this list of conditions and the following disclaimer.
>  *
>  * 2. Redistributions in binary form must reproduce the above copyright
>  *    notice, this list of conditions and the following disclaimer in
>  *    the documentation and/or other materials provided with the
>  *    distribution.
>  *
>  * 3. The end-user documentation included with the redistribution, if
>  *    any, must include the following acknowlegement:
>  *       "This product includes software developed by the
>  *        Apache Software Foundation (http://www.apache.org/)."
>  *    Alternately, this acknowlegement may appear in the software 
itself,
>  *    if and wherever such third-party acknowlegements normally appear.
>  *
>  * 4. The names "The Jakarta Project", "HttpClient", and "Apache 
Software
>  *    Foundation" must not be used to endorse or promote products 
derived
>  *    from this software without prior written permission. For written
>  *    permission, please contact apache@apache.org.
>  *
>  * 5. Products derived from this software may not be called "Apache"
>  *    nor may "Apache" appear in their names without prior written
>  *    permission of the Apache Group.
>  *
>  * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED
>  * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
>  * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
>  * DISCLAIMED.  IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION OR
>  * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
>  * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
>  * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
>  * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
>  * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
>  * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
>  * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
>  * SUCH DAMAGE.
>  * ====================================================================
>  *
>  * This software consists of voluntary contributions made by many
>  * individuals on behalf of the Apache Software Foundation.  For more
>  * information on the Apache Software Foundation, please see
>  * <http://www.apache.org/>.
>  *
>  * [Additional notices, if required by prior licensing conditions]
>  *
>  */
> 
> package org.apache.commons.httpclient;
> 
> import java.io.*;
> 
> /**
>  * Cuts the wrapped InputStream off after a specified number of bytes.
>  *
>  * @author Ortwin Glück
>  * @since 2.0
>  */
> 
> public class ContentLengthInputStream extends FilterInputStream {
>     private int contentLength;
>     private int pos = 0;
> 
>     /**
>      * Creates a new length limited stream
>      *
>      * @param in The stream to wrap
>      * @param contentLength The maximum number of bytes that can be read 
from
>      * the stream. Subsequent read operations will return -1.
>      */
>     public ContentLengthInputStream(InputStream in, int contentLength) {
>         super(in);
>         this.contentLength = contentLength;
>     }
> 
>     public int read() throws java.io.IOException {
>         if (pos > contentLength) return -1;
>         pos++;
>         return super.read();
>     }
> 
> 
>     public int read(byte[] b, int off, int len) throws 
java.io.IOException {
>         if (pos > contentLength) return -1;
>         if (pos + len > contentLength) {
>             len = contentLength - pos;
>         }
>         int count = super.read(b, off, len);
>         pos += count;
>         return count;
>     }
> 
> 
>     public int read(byte[] b) throws java.io.IOException {
>         return read(b, 0, b.length);
>     }
> 
> }package org.apache.commons.httpclient;
> 
> import java.io.*;
> import org.apache.commons.logging.*;
> 
> /**
>  * Logs all data read to the wire log.
>  *
>  * @author Ortwin Glück
>  * @since 2.0
>  */
> 
> public class WireLogInputStream extends FilterInputStream {
>     /** Log for any wire messages. */
>     private static final Log wireLog = 
LogFactory.getLog("httpclient.wire");
> 
>     public WireLogInputStream(InputStream in) {
>         super(in);
>     }
> 
>     public int read(byte[] b, int off, int len) throws 
java.io.IOException {
>         int l = super.read(b,  off,  len);
>         wireLog.debug("<< "+ new String(b, off, len));
>         return l;
>     }
> 
>     public int read() throws java.io.IOException {
>         int l = super.read();
>         if (l > 0) wireLog.debug("<< "+ (char) l);
>         return l;
>     }
> 
>     public int read(byte[] b) throws java.io.IOException {
>         int l = super.read(b);
>         wireLog.debug("<< "+ new String(b));
>         return l;
>     }
> }
> package org.apache.commons.httpclient;
> 
> import java.io.*;
> import java.util.Map;
> import junit.framework.*;
> 
> 
> public class TestStreams extends TestCase {
> 
>     public TestStreams(String testName) {
>         super(testName);
>     }
> 
>     public void testChunkedInputStream() throws IOException {
>         String correct = "10; this is 
> comments\r\n1234567890123456\r\n5\r\n12345\r\n0\r\nFooter1: 
> abcde\r\nFooter2: fghij\r\n";
>         HttpMethod method = new SimpleHttpMethod();
>         InputStream in = new ChunkedInputStream(new 
> ByteArrayInputStream(correct.getBytes()), method);
>         byte[] buffer = new byte[300];
>         ByteArrayOutputStream out = new ByteArrayOutputStream();
>         int len;
>         while ((len = in.read(buffer)) > 0) {
>             out.write(buffer, 0, len);
>         }
>         String result = new String(out.toByteArray());
>         assertEquals(result, "123456789012345612345");
>         Header footer = method.getResponseFooter("footer1");
>         assertEquals(footer.getValue(), "abcde");
>         footer = method.getResponseFooter("footer2");
>         assertEquals(footer.getValue(), "fghij");
>     }
> 
>     public void testContentLengthInputStream() throws IOException {
>         String correct = "1234567890123456";
>         InputStream in = new ContentLengthInputStream(new 
> ByteArrayInputStream(correct.getBytes()), 10);
>         byte[] buffer = new byte[50];
>         int len = in.read(buffer);
>         ByteArrayOutputStream out = new ByteArrayOutputStream();
>         out.write(buffer, 0, len);
>         String result = new String(out.toByteArray());
>         assertEquals(result, "1234567890");
>     }
> 
>     // ------------------------------------------------------- 
> TestCase Methods
> 
>     public static Test suite() {
>         return new TestSuite(TestHeader.class);
>     }
> 
>     // 
> ------------------------------------------------------------------- Main
>     public static void main(String args[]) {
>         String[] testCaseName = { TestStreams.class.getName() };
>         junit.textui.TestRunner.main(testCaseName);
>     }
> }
> ? java/org/apache/commons/httpclient/AutoCloseInputStream.java
> ? java/org/apache/commons/httpclient/ChunkedInputStream.java
> ? java/org/apache/commons/httpclient/ContentLengthInputStream.java
> ? java/org/apache/commons/httpclient/WireLogInputStream.java
> ? test/org/apache/commons/httpclient/TestStreams.java
> Index: java/org/apache/commons/httpclient/Authenticator.java
> ===================================================================
> RCS file: /home/cvspublic/jakarta-
> 
commons/httpclient/src/java/org/apache/commons/httpclient/Authenticator.java,v
> retrieving revision 1.25
> diff -u -w -r1.25 Authenticator.java
> Index: java/org/apache/commons/httpclient/HttpClient.java
> ===================================================================
> RCS file: /home/cvspublic/jakarta-
> 
commons/httpclient/src/java/org/apache/commons/httpclient/HttpClient.java,v
> retrieving revision 1.54
> diff -u -w -r1.54 HttpClient.java
> Index: java/org/apache/commons/httpclient/HttpConnection.java
> ===================================================================
> RCS file: /home/cvspublic/jakarta-
> 
commons/httpclient/src/java/org/apache/commons/httpclient/HttpConnection.
> java,v
> retrieving revision 1.18
> diff -u -w -r1.18 HttpConnection.java
> --- java/org/apache/commons/httpclient/HttpConnection.java   20 Aug 
> 2002 23:22:48 -0000   1.18
> +++ java/org/apache/commons/httpclient/HttpConnection.java   28 Aug 
> 2002 10:31:38 -0000
> @@ -457,7 +457,7 @@
>      throws IOException, IllegalStateException {
>          log.trace("enter 
HttpConnection.getRequestOutputStream(HttpMethod)");
>          assertOpen();
> -        return new ResponseInputStream(_input,method);
> +        return _input;
>      }
> 
>      /**
> Index: java/org/apache/commons/httpclient/HttpMethod.java
> ===================================================================
> RCS file: /home/cvspublic/jakarta-
> 
commons/httpclient/src/java/org/apache/commons/httpclient/HttpMethod.java,v
> retrieving revision 1.14
> diff -u -w -r1.14 HttpMethod.java
> --- java/org/apache/commons/httpclient/HttpMethod.java   28 Jul 2002
> 18:08:57 -0000   1.14
> +++ java/org/apache/commons/httpclient/HttpMethod.java   28 Aug 2002
> 10:31:38 -0000
> @@ -239,6 +239,18 @@
>      public Header getResponseHeader(String headerName);
> 
>      /**
> +     * Return an array of my response footers
> +     * @return <tt>null</tt> if no footers are available
> +     */
> +    public Header[] getResponseFooters();
> +
> +    /**
> +     * Return the specified response footer.
> +     * Note that footer-name matching is case insensitive.
> +     */
> +    public Header getResponseFooter(String footerName);
> +
> +    /**
>       * Return my response body, if any,
>       * as a byte array.
>       * Otherwise return <tt>null</tt>.
> @@ -286,4 +298,9 @@
>       * once this method has been called.
>       */
>      public void recycle();
> +
> +    /**
> +     * Use this method internally to add footers.
> +     */
> +    public void addResponseFooter(Header footer);
>  }
> Index: java/org/apache/commons/httpclient/HttpMethodBase.java
> ===================================================================
> RCS file: /home/cvspublic/jakarta-
> 
commons/httpclient/src/java/org/apache/commons/httpclient/HttpMethodBase.
> java,v
> retrieving revision 1.47
> diff -u -w -r1.47 HttpMethodBase.java
> --- java/org/apache/commons/httpclient/HttpMethodBase.java   14 Aug 
> 2002 01:41:58 -0000   1.47
> +++ java/org/apache/commons/httpclient/HttpMethodBase.java   28 Aug 
> 2002 10:31:40 -0000
> @@ -73,6 +73,7 @@
>  import java.net.MalformedURLException;
>  import java.net.URL;
>  import java.util.Date;
> +import java.util.Map;
>  import java.util.HashMap;
>  import java.util.HashSet;
>  import java.util.Iterator;
> @@ -304,6 +305,14 @@
>      }
> 
>      /**
> +     * adds a response footer to the internal list
> +     */
> +    public void addResponseFooter(Header footer) {
> +        if (responseFooters == null) responseFooters = new HashMap();
> +        responseFooters.put(footer.getName().toLowerCase(), footer);
> +    }
> +
> +    /**
>       * Whether or not I should automatically follow
>       * HTTP redirects (status code 302, etc.)
>       * @return <tt>true</tt> if I will automatically follow HTTP 
redirects
> @@ -431,11 +440,57 @@
>      }
> 
>      /**
> +     * Return an array of response footers.
> +     * @return <tt>null</tt> if no footers are available
> +     */
> +    public Header[] getResponseFooters() {
> +        if (responseFooters == null) {
> +            return null;
> +        }
> +        return (Header[])(responseFooters.values().toArray(
> +            new Header[responseFooters.size()]));
> +    }
> +
> +    /**
> +     * Get the response footer associated with the given name.
> +     * Footer name matching is case insensitive.
> +     * <tt>null</tt> will be returned if either <i>footerName</i>
is
> +     * <tt>null</tt> or there is no matching header for 
<i>footerName</i>
> +     * or there are no footers available.
> +     * @param footerName the footer name to match
> +     * @return the matching footer
> +     */
> +    public Header getResponseFooter(String footerName) {
> +        if (responseFooters == null) {
> +            return null;
> +        }
> +        return (footerName == null) ? null :
> +            (Header)(responseFooters.get(footerName.toLowerCase()));
> +    }
> +
> +    /**
>       * Return my response body, if any,
>       * as a byte array.
>       * Otherwise return <tt>null</tt>.
>       */
>      public byte[] getResponseBody() {
> +        if (responseBody == null) {
> +            try {
> +                ByteArrayOutputStream os = new ByteArrayOutputStream();
> +                InputStream is = getResponseBodyAsStream();
> +                byte[] buffer = new byte[10000];
> +                int len;
> +                while ((len = is.read(buffer)) > 0) {
> +                    os.write(buffer, 0, len);
> +                }
> +                responseBody = os.toByteArray();
> +                setResponseStream(null);
> +                log.debug("buffering response body");
> +            } catch(IOException e) {
> +                log.error("getResponseBody failed", e);
> +                responseBody = null;
> +            }
> +        }
>          return responseBody;
>      }
> 
> @@ -445,7 +500,15 @@
>       * Otherwise return <tt>null</tt>.
>       */
>      public String getResponseBodyAsString() {
> -        return null == responseBody ? null : new String(responseBody);
> +        return responseAvailable() ? new String(getResponseBody()) : 
null;
> +    }
> +
> +    /**
> +     * Checks if response data is available.
> +     * @return true if response data is available, false otherwise.
> +     */
> +    private boolean responseAvailable() {
> +        return (responseBody != null) || (responseStream != null);
>      }
> 
>      /**
> @@ -454,8 +517,19 @@
>       * Otherwise return <tt>null</tt>.
>       */
>      public InputStream getResponseBodyAsStream() throws IOException {
> -        return null == responseBody ? null : 
> -            new ByteArrayInputStream(responseBody);
> +        if (responseStream != null) {
> +            return responseStream;
> +        }
> +        if (responseBody != null) {
> +            responseStream = new ByteArrayInputStream(responseBody);
> +            log.debug("re-creating response stream from byte array");
> +            return responseStream;
> +        }
> +        return null;
> +    }
> +
> +    protected void setResponseStream(InputStream responseStream) {
> +        this.responseStream = responseStream;
>      }
> 
>      /**
> @@ -557,32 +631,42 @@
>      }
> 
>      /**
> +     * Close the provided HTTP connection if nccessary
> +     * @param connection the HTTP connection to process
> +     */
> +    private void closeConnection(HttpConnection connection) {
> +        if (shouldCloseConnection(connection)) {
> +            connection.close();
> +        }
> +    }
> +
> +    /**
>       * Close the provided HTTP connection, if:
>       *  http 1.0 and not using the 'connect' method, or
>       *  http 1.1 and the Connection: close header is sent
>       *
>       * @param connection the HTTP connection to process
>       */
> -    private void closeConnection(HttpConnection connection) {
> -        log.trace("enter closeConnection(HttpConnection)");
> -
> +    private boolean shouldCloseConnection(HttpConnection connection) {
>          if (!http11) {
>              if (getName().equals(ConnectMethod.NAME) && 
>                      (statusCode == HttpStatus.SC_OK)) {
> -                log.debug("Leaving connection open for tunneling");
> +                log.debug("Will leave connection open for tunneling");
> +                return false;
>              } else {
> -                log.debug("Closing connection since using HTTP/1.0, " + 

> +                log.debug("Should close connection since using 
HTTP/1.0, " +
>                          "ConnectMethod and status is OK");
> -                connection.close();
> +                return true;
>              }
>          } else {
>              Header connectionHeader = getResponseHeader("connection");
>              if (null != connectionHeader
>                  && 
"close".equalsIgnoreCase(connectionHeader.getValue())) {
> -                log.debug("Closing connection since \"Connection: 
> close\" header found.");
> -                connection.close();
> +                log.debug("Should close connection since 
> \"Connection: close\" header found.");
> +                return true;
>              }
>          }
> +        return false;
>      }
> 
> 
> @@ -743,8 +827,7 @@
>              //if SC_CONTINUE write the request body
>              writeRemainingRequestBody(state, conn);
> 
> -            //close connection if required
> -            closeConnection(conn);
> +
> 
>              switch(statusCode){
>                  case HttpStatus.SC_UNAUTHORIZED:
> @@ -849,7 +932,8 @@
>                  return statusCode;
>              }
>              visited.add(generateVisitedKey(conn));
> -
> +            //close connection if required
> +            closeConnection(conn);
>          }//end of loop
> 
>          log.error("Narrowly avoided an infinite loop in execute");
> @@ -1010,7 +1094,7 @@
> 
>          if (isIpAddress(host)) {
>              log.debug("Adding empty Host request header: host is an
> ipaddress");
> -            setRequestHeader("Host", null);
> +            setRequestHeader("Host", "");
>              return;
>          }
> 
> @@ -1396,10 +1480,8 @@
>      /**
>       * Read the response body from the given {@link HttpConnection}.
>       * <p>
> -     * The current implementation simply consumes the expected
> -     * response body (according to the values of the
> -     * <tt>Content-Length</tt> and <tt>Transfer-Encoding</tt>
> -     * headers, if any).
> +     * The current implementation simply gets the stream reference
> +     * from the connection.
>       * <p>
>       * Subclasses may want to override this method to
>       * to customize the processing.
> @@ -1414,88 +1496,56 @@
>      throws IOException, HttpException {
>          log.trace("enter HttpMethodBase.readResponseBody(HttpState,
> HttpConnection)");
> 
> -        ByteArrayOutputStream out = new ByteArrayOutputStream();
> -        readResponseBody(state, conn, out);
> -
> -        out.close();
> -        responseBody = out.toByteArray();
> +        setResponseStream(_readResponseBody(state, conn));
>      }
> 
>      /**
>       * Read the response body from the given {@link HttpConnection}.
>       * <p>
> -     * The current implementation simply consumes the expected
> -     * response body (according to the values of the
> +     * The current implementation returns an appropriate stream
> +     * (according to the values of the
>       * <tt>Content-Length</tt> and <tt>Transfer-Encoding</tt>
>       * headers, if any).
>       * <p>
> -     * Subclasses may want to override this method to
> -     * to customize the processing.
>       *
>       * @see #readResponse
>       * @see #processResponseBody
>       *
>       * @param state the client state
>       * @param conn the {@link HttpConnection} to read the response from
> -     * @param out OutputStream to write the response body to
> +     * @returns InputStream to read the response body from
>       */
> -    protected void readResponseBody(HttpState state, HttpConnection 
conn, 
> -    OutputStream out) throws IOException {
> +    private InputStream _readResponseBody(HttpState state, 
> HttpConnection conn) throws IOException {
>          log.trace("enter HttpMethodBase.readResponseBody(HttpState,
> HttpConnection)");
> 
> -        responseBody = null;
> -        int expectedLength = 0;
> -        int foundLength = 0;
> -        {
> +        responseBody = null; // is this desired?
>              Header lengthHeader = getResponseHeader("Content-Length");
>              Header transferEncodingHeader = 
> getResponseHeader("Transfer-Encoding");
> +        InputStream is = conn.getResponseInputStream(this);
> +        if (wireLog.isDebugEnabled()) {
> +            is = new WireLogInputStream(is);
> +        }
> +        InputStream result = null;
>              if (null != lengthHeader) {
>                  try {
> -                    expectedLength = Integer.parseInt(lengthHeader.
> getValue());
> +                int expectedLength = Integer.parseInt(lengthHeader.
> getValue());
> +                result = new ContentLengthInputStream(is, 
expectedLength);
>                  } catch(NumberFormatException e) {
>                      // ignored
>                  }
>              } else if (null != transferEncodingHeader) {
>                  if ("chunked".
> equalsIgnoreCase(transferEncodingHeader.getValue())) {
> -                    expectedLength = -1;
> +                result = new ChunkedInputStream(is, this);
>                  }
>              } else if(canResponseHaveBody(statusCode) && !
> getName().equals(ConnectMethod.NAME)){
> -                /*
> -                 * According to the specification, a response with 
> neither Content-Length
> -                 * nor Transfer-Encoding indicates that the 
> response has no body.  In
> -                 * the real world, this usually means that the 
> server just didn't know
> -                 * the content-length when it sent the response. 
> FIXME:  Should we do
> -                 * this only in non-strict mode?
> -                 * If we do this we will hang forever waiting for a 
body!
> -                 */
> -                expectedLength = -1;
> -            }
> -        }
> -        InputStream is = conn.getResponseInputStream(this);
> -        byte[] buffer = new byte[4096];
> -        int nb = 0;
> -        while(expectedLength == -1 || foundLength < expectedLength) {
> -            nb = is.read(buffer);
> -            if (nb == -1) {
> -                break;
> -            }
> -            if (out == null) {
> -                throw new IOException("Unable to buffer data");
> -            }
> -            if (wireLog.isDebugEnabled()) {
> -                wireLog.debug("<< \"" + new String(buffer, 0, nb) + 
"\"");
> -            }
> -            out.write(buffer, 0, nb);
> -            foundLength += nb;
> -            if (expectedLength > -1) {
> -                if (foundLength == expectedLength) {
> -                    break;
> -                } else if (foundLength > expectedLength) {
> -                    log.warn("HttpMethodBase.readResponseBody(): 
> expected length (" + expectedLength + ") exceeded.  Found " + 
> foundLength + " bytes.");
> -                    break;
> -                }
> +            result = is;
>              }
> +        if (result == null) return null;
> +
> +        if (shouldCloseConnection(conn)) {
> +            result = new AutoCloseInputStream(result, conn);
>          }
> +        return result;
>      }
> 
>      /**
> @@ -1688,6 +1738,8 @@
>      private HashMap requestHeaders = new HashMap();
>      /** My response headers, if any. */
>      private HashMap responseHeaders = new HashMap();
> +    /** My response footers, if any. */
> +    private Map responseFooters = null;
>      /** My response status code, if any. */
>      private int statusCode = -1;
>      /** My response status text, if any. */
> @@ -1699,6 +1751,8 @@
>      /** Whether or not the request body has been sent. */
>      private boolean bodySent = false;
>      /** The response body, assuming it has not be intercepted by a 
> sub-class. */
> +    private InputStream responseStream = null;
> +    /** Buffer for the response */
>      private byte[] responseBody = null;
>      /** The maximum number of attempts to attempt recovery from an 
> HttpRecoverableException. */
>      private int maxRetries = 3;
> Index: java/org/apache/commons/httpclient/HttpMultiClient.java
> ===================================================================
> RCS file: /home/cvspublic/jakarta-
> 
commons/httpclient/src/java/org/apache/commons/httpclient/HttpMultiClient.
> java,v
> retrieving revision 1.13
> diff -u -w -r1.13 HttpMultiClient.java
> Index: java/org/apache/commons/httpclient/methods/GetMethod.java
> ===================================================================
> RCS file: /home/cvspublic/jakarta-
> 
commons/httpclient/src/java/org/apache/commons/httpclient/methods/GetMethod.
> java,v
> retrieving revision 1.15
> diff -u -w -r1.15 GetMethod.java
> --- java/org/apache/commons/httpclient/methods/GetMethod.java   8 
> Aug 2002 21:51:36 -0000   1.15
> +++ java/org/apache/commons/httpclient/methods/GetMethod.java   28 
> Aug 2002 10:31:41 -0000
> @@ -65,6 +65,7 @@
>  import org.apache.commons.httpclient.HttpConnection;
>  import org.apache.commons.httpclient.HttpMethodBase;
>  import org.apache.commons.httpclient.HttpState;
> +import org.apache.commons.httpclient.HttpException;
>  import org.apache.commons.logging.Log;
>  import org.apache.commons.logging.LogFactory;
> 
> @@ -181,12 +182,6 @@
> 
> 
>      /**
> -     * If we're not using the HD, we're using a memory byte buffer.
> -     */
> -    protected byte[] memoryData;
> -
> -
> -    /**
>       * File which contains the buffered data.
>       */
>      protected File fileData;
> @@ -323,7 +318,6 @@
>          log.trace("enter GetMethod.recycle()");
> 
>          super.recycle();
> -        this.memoryData = null;
>          this.fileData = null;
>          setFollowRedirects(true);
>      }
> @@ -339,45 +333,7 @@
>         log.trace("enter GetMethod.getResponseBody()");
> 
>         checkUsed();
> -       if(useDisk) {
> -           try {
> -               InputStream is = new FileInputStream(fileData);
> -               byte[] buffer = new byte[4096];
> -               ByteArrayOutputStream os = new ByteArrayOutputStream();
> -               int nb = 0;
> -               while (true) {
> -                   nb = is.read(buffer);
> -                   if (nb == -1)
> -                       break;
> -                   os.write(buffer, 0, nb);
> -               }
> -               is.close();
> -               return os.toByteArray();
> -           } catch(IOException e) {
> -               log.error("Exception in GetMethod.getResponseBody() 
> while retrieving data from file \"" + fileData + "\".",e);
> -               return null;
> -           }
> -       } else {
> -           return memoryData;
> -       }
> -   }
> -
> -   /**
> -    * Return my response body, if any,
> -    * as a {@link String}.
> -    * Otherwise return <tt>null</tt>.
> -     *
> -     * @since 2.0
> -    */
> -   public String getResponseBodyAsString() {
> -       log.trace("enter GetMethod.getResponseBodyAsString()");
> -
> -       byte[] data = getResponseBody();
> -       if(null == data) {
> -           return null;
> -       } else {
> -           return new String(data);
> -       }
> +       return super.getResponseBody();
>     }
> 
> 
> @@ -393,15 +349,7 @@
>          log.trace("enter GetMethod.getResponseBodyAsStream()");
> 
>          checkUsed();
> -        if (useDisk) {
> -            return new FileInputStream(fileData);
> -        } else {
> -            if(null == memoryData) {
> -                return null;
> -            } else {
> -                return new ByteArrayInputStream(memoryData);
> -            }
> -        }
> +        return super.getResponseBodyAsStream();
>      }
> 
> 
> @@ -414,11 +362,30 @@
>       * @since 2.0
>       */
>      protected void readResponseBody(HttpState state, HttpConnection 
conn)
> -    throws IOException {
> +    throws IOException, HttpException {
>          log.trace("enter GetMethod.readResponseBody(HttpState, 
> HttpConnection)");
> +        super.readResponseBody(state, conn);
> 
>          OutputStream out = null;
>          if (useDisk) {
> +            out = new FileOutputStream(createTempFile());
> +            InputStream in = getResponseBodyAsStream();
> +            byte[] buffer = new byte[10000];
> +            int len ;
> +            while ((len = in.read(buffer)) > 0) {
> +                out.write(buffer, 0, len);
> +            }
> +            out.close();
> +            setResponseStream(new FileInputStream(createTempFile()));
> +        }
> +    }
> +
> +    /**
> +     * Returns the file buffer, creating it if necessary. The created 
file is
> +     * deleted when the VM exits.
> +     * @return Temporary file to hold the data buffer.
> +     */
> +    private File createTempFile() {
>              if (fileData == null) {
>                  // Create a temporary file on the HD
>                  File dir = new File(tempDir);
> @@ -440,17 +407,6 @@
>                  fileData = new File(tempDir, tempFileName);
>                  fileData.deleteOnExit();
>              }
> -            out = new FileOutputStream(fileData);
> -        } else {
> -            out = new ByteArrayOutputStream();
> -        }
> -
> -        readResponseBody(state, conn, out);
> -
> -        if (!useDisk) {
> -            memoryData = ((ByteArrayOutputStream) out).toByteArray();
> -        }
> -
> -        out.close();
> +        return fileData;
>      }
>  }
> Index: test/org/apache/commons/httpclient/TestMethodsNoHost.java
> ===================================================================
> RCS file: /home/cvspublic/jakarta-
> 
commons/httpclient/src/test/org/apache/commons/httpclient/TestMethodsNoHost.
> java,v
> retrieving revision 1.7
> diff -u -w -r1.7 TestMethodsNoHost.java
> Index: test/org/apache/commons/httpclient/TestNoHost.java
> ===================================================================
> RCS file: /home/cvspublic/jakarta-
> 
commons/httpclient/src/test/org/apache/commons/httpclient/TestNoHost.java,v
> retrieving revision 1.10
> diff -u -w -r1.10 TestNoHost.java
> --- test/org/apache/commons/httpclient/TestNoHost.java   2 Aug 2002 
> 11:38:12 -0000   1.10
> +++ test/org/apache/commons/httpclient/TestNoHost.java   28 Aug 2002
> 10:31:44 -0000
> @@ -97,6 +97,7 @@
>          suite.addTest(TestHttpState.suite());
>          suite.addTest(TestResponseHeaders.suite());
>          suite.addTest(TestRequestHeaders.suite());
> +        suite.addTest(TestStreams.suite());
>          return suite;
>      }
> 
> Index: test/org/apache/commons/httpclient/TestWebappHeaders.java
> ===================================================================
> RCS file: /home/cvspublic/jakarta-
> 
commons/httpclient/src/test/org/apache/commons/httpclient/TestWebappHeaders.
> java,v
> retrieving revision 1.4
> diff -u -w -r1.4 TestWebappHeaders.java
> --- test/org/apache/commons/httpclient/TestWebappHeaders.java   13 
> Jul 2002 09:07:01 -0000   1.4
> +++ test/org/apache/commons/httpclient/TestWebappHeaders.java   28 
> Aug 2002 10:31:46 -0000
> @@ -205,7 +205,7 @@
>          client.endSession();
>          Header hostHeader = get.getRequestHeader("Host");
>          assertTrue(hostHeader != null);
> -        assertTrue(hostHeader.getValue().equals(""));
> +        assertTrue(hostHeader.getValue() == null);
> 
>          // reset 
>          get.recycle();
> 
> --
> To unsubscribe, e-mail: 
<mailto:commons-dev-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: 
<mailto:commons-dev-help@jakarta.apache.org>

--
To unsubscribe, e-mail:   <mailto:commons-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:commons-dev-help@jakarta.apache.org>


Mime
View raw message