hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roland Weber <ROLWE...@de.ibm.com>
Subject RE: HttpClient bandwidth
Date Wed, 31 Aug 2005 05:35:33 GMT
Hi B. K.

> I am a little confused by your comments Gus as I understand it the 
request 
> headers are being sent as part of the request to the server, how would 
> checking these headers help me out?

He meant the response headers. Also keep in mind that closing the 
connection
might not be enough, because of the keep-alive mechanism used by 
HttpClient.
You can use HttpMethod.abort() to make sure the response body is not 
received,
or you can send a "Connection: close" header in the request.

But your first try should still be what Oleg suggested, using headers
If-Modified-Since and/or ETag. If that works, it would not only reduce the
bandwith consumed, but also the load on the server.

cheers,
  Roland

> 
> >From: Gustavo Hexsel <ghexsel@sagebrushcorp.com>
> >Reply-To: "HttpClient User Discussion" 
<httpclient-user@jakarta.apache.org>
> >To: 'HttpClient User Discussion' <httpclient-user@jakarta.apache.org>
> >Subject: RE: HttpClient bandwidth
> >Date: Tue, 30 Aug 2005 10:25:51 -0500
> >
> >   Can't you just check the header's value with getRequestHeader() 
before 
> >you
> >call getResponseBodyAsStream()?
> >
> >   If it doesn't match, just close the connection... the website will
> >generate the response page anyway and the load should be about the same 
for
> >it, but the httpClient doesn't need all that.
> >
> >   []s Gus
> >
> >
> >-----Original Message-----
> >From: Oleg Kalnichevski [mailto:olegk@apache.org]
> >Sent: August 30, 2005 6:24 AM
> >To: httpclient-user@jakarta.apache.org
> >Subject: Re: HttpClient bandwidth
> >
> >
> >On Tue, Aug 30, 2005 at 10:15:50PM +1000, B K wrote:
> > > Hi all,
> > >     I have developed an application using httpclient, and now that I
> > >     have started using it, it is using to much bandwidth I wonder if
> >anybody
> > > has some pointers on how to reduce the amount of data being 
transferred.
> >My
> > > idea was to use the response headers to only retrieve the response 
if 
> >the
> > > data had changed, unfortunately I can't find any way of doing this, 
my
> >idea
> > > was to only retrieve the response when the web content changed. The 
web
> > > sites I am dealing with are very dynamic and change every couple of
> >minutes
> > > so I need to check every minute for updates to the data.
> > >
> > > The application is gathering data from 10 web sites, and I have 
> >developed
> > > it so there are 10 instances of httpclient, one for each web site, 
the 
> >big
> >
> > > problem I see is everytime I send out the requests I have to 
download 
> >the
> > > response even if there is no change in the data. Anybody got some 
bright
> > > design ideas on how to cut down, I have researched and come up with
> >nothing.
> > >
> > > Thanks
> > >
> >
> >B K,
> >
> >There's not much you can do unless the target servers play along. 
Please
> >refer to the HTTP spec [1] and take a look at the 304 Not Modified
> >mechanism [2] for details
> >
> >Hope this helps
> >
> >Oleg
> >[1] http://www.w3.org/Protocols/rfc2616/rfc2616.html
> >[2] http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Mime
View raw message