hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Becke <mbe...@gmail.com>
Subject Re: Making ContentLengthInputStream stop when I want, is this safe?
Date Sat, 02 Jul 2005 23:56:40 GMT
Hi Jim,

HttpMethod.abort() is probably what you're looking for.

Your workaround solution could potentially cause some issues.  In
particular it will be a problem if you end up reusing a connection
that has been skipped.  The remaining content will still be on the
wire and it will be read the next time the connection is used. 
Reading to the end of the response is specifically done so that
connections can be reused.  In your case it sounds like it would be
better to abort the method, which causes the connection to be closed
and not reused.

Mike

On 7/2/05, Jim Roepcke <jim@roepcke.com> wrote:
> Hi,
> 
> I'm writing an application where the user must be able to interrupt
> and resume downloads at will.
> 
> I was noticing noticeable pauses when I called releaseConnection() on
> the GetMethod that I was reading the response from.  Recently I
> realized this is because deep in HttpClient, releaseConnection
> eventually calls exhaustInputStream to read and discard the rest of
> the response (!!!) before closing the connection.  Since the file
> downloads the user will interrupt are potentially many gigabytes in
> size, having releaseConnection block while it reads and discards many
> gigabytes of data is obviously very bad.
> 
> I created a workaround.  It seems to be working for me, ie:
> releaseConnection is no longer pausing, but I am guessing the
> developers of HttpClient had something important in mind when they
> decided to FORCE responses to be COMPLETELY read.  Could you please
> let me know if my hack is going to cause bad side-effects?
> 
> Background info: I'm using the MultiThreadedHttpConnectionManager.
> At any time I could have up to one (significant) GET download, one
> (significant) PUT upload, and possibly a very small number of very
> short DELETE, HEAD and GET requests executing.
> 
> The hack:
> 
> In ContentLengthInputStream, I added a method forceSkipTheRest().  It
> does the following:
> 
>      public void forceSkipTheRest() {
>          this.contentLength = this.pos;
>      }
> 
> In AutoCloseInputStream, I added an accessor method to get the
> wrapped [ContentLength]InputStream, so that I can call the
> forceSkipTheRest method on it.
> 
> Here is a snippet of code from my Download worker code (which runs in
> a separate thread managed by Doug Lea's util.concurrent's
> QueuedExectutor class).  Please refer to the section near the bottom
> inside the if (shouldStop()) block for the use of the gross hack...
> 
> // [snip]
> client.executeMethod(hc, get);
> int status_code_get = get.getStatusCode();
> 
> if ((status_code_get / 100) == 2) { // in the 200 series of responses
>      responseStream = get.getResponseBodyAsStream();
>      int download_stream_buf_size = 4096;
>      // TODO: review again if this bufferedinputstream is needed
>      // it seems it is redundant, the responseStream is probably good
> enough!!
>      downloadStream = new BufferedInputStream(responseStream,
> download_stream_buf_size);
>      byte[] downloadBuffer = new byte[download_stream_buf_size];
> 
>      int bytesRead = 0;
>      raf = new RandomAccessFile(downloadFile, "rw");
>      while ((bytesRead = downloadStream.read(downloadBuffer, 0,
> download_stream_buf_size)) > 0) {
>          raf.write(downloadBuffer, 0, bytesRead);
>          totalBytesRead += bytesRead;
>          if (shouldStop()) {
>              if (responseStream instanceof AutoCloseInputStream) {
>                  InputStream stream = ((AutoCloseInputStream)
> responseStream).getWrappedInputStream();
>                  if (stream instanceof ContentLengthInputStream)
>                      ((ContentLengthInputStream)
> stream).forceSkipTheRest();
>              }
>              break;
>          }
>      }
>      get.releaseConnection();
>      get = null;
>      // [snip]
> 
> In some quick testing, it seems this is working just fine... however,
> I wonder if any bad side effects will result from this.
> 
> I'm also wondering if perhaps I should simply set a flag in
> ContentLengthInputStream, and if that flag is true, when close() is
> called on ContentLegnthInputStream it calls forceSkipTheRest before
> calling exhaustInputStream (so that this interruption works the way I
> need in all cases, not just in my read loop).
> 
> Your advice is *greatly* appreciated!!
> 
> Thanks,
> 
> Jim
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> 
>

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Mime
View raw message