hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Burbidge <mburb...@adobe.com>
Subject Httpclient 3.1 - reading the response from a GET request...
Date Wed, 23 May 2012 14:42:20 GMT
I think I might have a fundamental misunderstanding about how httpclient works.

Suppose I have set up a get request using the following code snippet:

	GetMethod getMethod = new GetMethod("https://somedownloadurl.com");
	getMethod.setQueryString(someQueryString);
	getMethod.setParams(methodParams);

Once I've created my get method request I execute it using the following snippet:

	in responseCode = httpClient.executeMethod(getMethod);

I get back a successful response code and so I get the input stream to read the body, in this
case the response body contains an image. I get the stream using the following call:

InputStream responseStream = getMethod.getResponseBodyAsStream();

My question is about reading the bytes back from the input stream. Our code used to do the
following to read the input stream:

int numRead;
byte[] chunk = new byte[BUFFER_SIZE];

while (responseStream.available())
{
    numRead = responseStream.read(chunk);
}

I'm leaving out what we do with the bytes, because my question is about reading the stream.

This code worked for a couple of years over several releases. Then suddenly during the testing
cycle in one of our releases the following call started returning 0.

responseStream.available()

I never was able to determine what changed. Both the server stack and client stack are pretty
complex.

So I read the documentation on available() and concluded that we were misusing it. That call
simply means that there are no bytes available without blocking. So I changed the code to
the following:

byte[] chunk = new byte[BUFFER_SIZE];

int numRead = responseStream.read(chunk);
while (numRead > 0)
{
    numRead = responseStream.read(chunk);
}

That works, but I'm puzzled by how it behaves. No matter what buffer size I give it, the read
always returns only 1448 bytes at a time. One could conclude that the server is not able to
keep up with the client. Even if I put the following sleep statement in the while loop, to
stall the client. I still get only 1448 bytes at a time.

byte[] chunk = new byte[BUFFER_SIZE];

int numRead = responseStream.read(chunk);
while (numRead > 0)
{
    Thread.sleep(500);
    numRead = responseStream.read(chunk);
}

If I do a tcpdump I can see that the packet size coming across the wire corresponds to the
size of data that I'm getting back in each read request on the stream. My assumption was that
somewhere in the client networking stack, data would be read asynchronously as it came over
the wire. And that when I did a read on the stream I would get back large chunks of data,
that had been accumulated in buffers at the lower layers of the stack.

Is that a complete misunderstanding of what happens? Do you have any idea what could cause
a change in the behavior of the available() call? Do you think these two symptoms are related?

Thanks,
Michael-
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message