hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: Problem Downloading Images
Date Mon, 08 Nov 2004 12:23:34 GMT
Michael,

Try this and see it makes any difference.

package test;

import java.io.IOException;

import org.apache.commons.httpclient.Header;
import org.apache.commons.httpclient.HttpConnection;
import org.apache.commons.httpclient.HttpException;
import org.apache.commons.httpclient.HttpState;
import org.apache.commons.httpclient.methods.GetMethod;

public class BrokenLenMethod extends GetMethod {

public BrokenLenMethod() {
  super();
}

public BrokenLenMethod(final String uri) {
  super(uri);
}
    
protected void readResponseHeaders(
  HttpState state, HttpConnection conn) throws IOException, HttpException {
    // Remove broken content length header(s)
    Header[] headers =
      getResponseHeaderGroup().getHeaders("Content-Length");
    for (int i = 0; i < headers.length; i++) {
      getResponseHeaderGroup().removeHeader(headers[i]);
    }
    // Force close connection
    getResponseHeaderGroup().addHeader(
      new Header("Connection", "close"));
    super.readResponseHeaders(state, conn);
    }
}

Hope this helps

Oleg

On Mon, Nov 08, 2004 at 01:37:00AM -0800, Michael Taft wrote:
> Oleg -
> Seems like the server is messing them up now and again. Is there a 
> workaround so that the image gets downloaded anyway?
> M.
> 
> Oleg Kalnichevski wrote:
> 
> >Michael,
> >
> >In the responseBodyConsumed method HttpClient tries to determine if the
> >connection is still good to be reused. It checks whether there's some
> >extraneous content past the declared response body. Apparently something
> >goes wrong there. Can you check if the Content-Length /
> >Transfer-Encoding headers are kosher? Possibly the server miscalculates
> >the size of the images
> >
> >Oleg
> >
> >On Sun, 2004-11-07 at 23:24, Michael Taft wrote:
> >
> >>Oleg -
> >>Interesting. Something is happening when I get this info message, 
> >>however, because those images aren't downloaded properly (they are cut 
> >>short) whereas all the other ones work fine. Any ideas?
> >>M.
> >>
> >>Oleg Kalnichevski wrote:
> >>
> >>
> >>>Michael,
> >>>
> >>>This is not a problem, but merely an info message. You can get rid of it
> >>>by reducing the log verbosity from INFO to WARN
> >>>
> >>>Oleg
> >>>
> >>>On Sun, 2004-11-07 at 04:37, Michael Taft wrote:
> >>>
> >>>
> >>>>Hello -
> >>>>I have a quick question. I am downloading images and writing them to
> >>>>disk using code that looks like this:
> >>>>
> >>>>
> >>>>-----------------------------------------------------------------------------------------------------

> >>>>
> >>>>
> >>>>String nextPic = "img01.jpg";
> >>>>          for (int i=1; i<=24; i++)
> >>>>             {
> >>>>            String iString = Integer.toString(i);
> >>>>                         if(i<10)iString = "0"+iString;
> >>>>              nextPic = "img" + iString + ".jpg";
> >>>>        // System.out.println("saved " + nextPic);
> >>>>              get.setURI(new URI("http://targetsite" + nextPic, false));
> >>>>              client.executeMethod( get );
> >>>>                       InputStream is = get.getResponseBodyAsStream();
> >>>>              BufferedInputStream bis = new BufferedInputStream( is );
> >>>>              FileOutputStream fos = new FileOutputStream( nextPic );
> >>>>              byte[] bytes = new byte[ 200000 ];//room for a 200K file
> >>>>              int count = bis.read( bytes );
> >>>>              while( count != -1 && count <= 200000 ) {
> >>>>               System.out.print( "-" );
> >>>>               fos.write( bytes, 0, count );
> >>>>               count = bis.read( bytes );
> >>>>              }
> >>>>              System.out.println("saved " + nextPic);
> >>>>              if( count != -1 ) {
> >>>>               fos.write( bytes, 0, count );
> >>>>                             }
> >>>>              fos.close();
> >>>>              bis.close();
> >>>>                                    }
> >>>>          System.out.println("saved" + nextPic);
> >>>>
> >>>>-------------------------------------------------------------------------------------

> >>>>
> >>>>
> >>>>
> >>>>In this example, the assumption is that there are 24 images, named in
> >>>>numberical order. The code works fine, except that every once in a
> >>>>while, I'm getting a "responseBodyConsumed" message like this one:
> >>>>
> >>>>---------------------
> >>>>Nov 6, 2004 7:10:53 PM org.apache.commons.httpclient.HttpMethodBase
> >>>>responseBodyConsumed
> >>>>INFO: Stream closed
> >>>>---------------------
> >>>>
> >>>>I've been trying to figure out what causes this problem, and how to fix
> >>>>it. Any help?
> >>>>
> >>>>Thanks,
> >>>>M.
> >>>>
> >>>>---------------------------------------------------------------------
> >>>>To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> >>>>For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> >>>>
> >>>
> >>>
> >>>
> >>>---------------------------------------------------------------------
> >>>To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> >>>For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> >>>
> >>>
> >
> >
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> >For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> >
> >
> 
> -- 
> Michael W. Taft
> Screenwriter/Producer
> 4614 Finley Avenue, #3
> Los Angeles, CA 90027
> (323)663-6042
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Mime
View raw message