hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gustavo Hexsel <ghex...@sagebrushcorp.com>
Subject RE: HttpClient bandwidth
Date Tue, 30 Aug 2005 15:25:51 GMT
  Can't you just check the header's value with getRequestHeader() before you
call getResponseBodyAsStream()?

  If it doesn't match, just close the connection... the website will
generate the response page anyway and the load should be about the same for
it, but the httpClient doesn't need all that.

  []s Gus


-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org]
Sent: August 30, 2005 6:24 AM
To: httpclient-user@jakarta.apache.org
Subject: Re: HttpClient bandwidth


On Tue, Aug 30, 2005 at 10:15:50PM +1000, B K wrote:
> Hi all,
> 	 I have developed an application using httpclient, and now that I 
> 	 have started using it, it is using to much bandwidth I wonder if
anybody 
> has some pointers on how to reduce the amount of data being transferred.
My 
> idea was to use the response headers to only retrieve the response if the 
> data had changed, unfortunately I can't find any way of doing this, my
idea 
> was to only retrieve the response when the web content changed. The web 
> sites I am dealing with are very dynamic and change every couple of
minutes 
> so I need to check every minute for updates to the data.
> 
> The application is gathering data from 10 web sites, and I have developed 
> it so there are 10 instances of httpclient, one for each web site, the big

> problem I see is everytime I send out the requests I have to download the 
> response even if there is no change in the data. Anybody got some bright 
> design ideas on how to cut down, I have researched and come up with
nothing.
> 
> Thanks
> 

B K,

There's not much you can do unless the target servers play along. Please
refer to the HTTP spec [1] and take a look at the 304 Not Modified
mechanism [2] for details

Hope this helps

Oleg
[1] http://www.w3.org/Protocols/rfc2616/rfc2616.html
[2] http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5

> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Mime
View raw message