hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Roepcke <...@roepcke.com>
Subject Making ContentLengthInputStream stop when I want, is this safe?
Date Sat, 02 Jul 2005 23:49:04 GMT

I'm writing an application where the user must be able to interrupt  
and resume downloads at will.

I was noticing noticeable pauses when I called releaseConnection() on  
the GetMethod that I was reading the response from.  Recently I  
realized this is because deep in HttpClient, releaseConnection  
eventually calls exhaustInputStream to read and discard the rest of  
the response (!!!) before closing the connection.  Since the file  
downloads the user will interrupt are potentially many gigabytes in  
size, having releaseConnection block while it reads and discards many  
gigabytes of data is obviously very bad.

I created a workaround.  It seems to be working for me, ie:  
releaseConnection is no longer pausing, but I am guessing the  
developers of HttpClient had something important in mind when they  
decided to FORCE responses to be COMPLETELY read.  Could you please  
let me know if my hack is going to cause bad side-effects?

Background info: I'm using the MultiThreadedHttpConnectionManager.   
At any time I could have up to one (significant) GET download, one  
(significant) PUT upload, and possibly a very small number of very  
short DELETE, HEAD and GET requests executing.

The hack:

In ContentLengthInputStream, I added a method forceSkipTheRest().  It  
does the following:

     public void forceSkipTheRest() {
         this.contentLength = this.pos;

In AutoCloseInputStream, I added an accessor method to get the  
wrapped [ContentLength]InputStream, so that I can call the  
forceSkipTheRest method on it.

Here is a snippet of code from my Download worker code (which runs in  
a separate thread managed by Doug Lea's util.concurrent's  
QueuedExectutor class).  Please refer to the section near the bottom  
inside the if (shouldStop()) block for the use of the gross hack...

// [snip]
client.executeMethod(hc, get);
int status_code_get = get.getStatusCode();

if ((status_code_get / 100) == 2) { // in the 200 series of responses
     responseStream = get.getResponseBodyAsStream();
     int download_stream_buf_size = 4096;
     // TODO: review again if this bufferedinputstream is needed
     // it seems it is redundant, the responseStream is probably good  
     downloadStream = new BufferedInputStream(responseStream,  
     byte[] downloadBuffer = new byte[download_stream_buf_size];

     int bytesRead = 0;
     raf = new RandomAccessFile(downloadFile, "rw");
     while ((bytesRead = downloadStream.read(downloadBuffer, 0,  
download_stream_buf_size)) > 0) {
         raf.write(downloadBuffer, 0, bytesRead);
         totalBytesRead += bytesRead;
         if (shouldStop()) {
             if (responseStream instanceof AutoCloseInputStream) {
                 InputStream stream = ((AutoCloseInputStream) 
                 if (stream instanceof ContentLengthInputStream)
     get = null;
     // [snip]

In some quick testing, it seems this is working just fine... however,  
I wonder if any bad side effects will result from this.

I'm also wondering if perhaps I should simply set a flag in  
ContentLengthInputStream, and if that flag is true, when close() is  
called on ContentLegnthInputStream it calls forceSkipTheRest before  
calling exhaustInputStream (so that this interruption works the way I  
need in all cases, not just in my read loop).

Your advice is *greatly* appreciated!!



To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org

View raw message