hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@apache.org>
Subject Re: Forcefully closing ContentLengthInputStream: is it possible?
Date Wed, 05 Oct 2011 06:38:22 GMT
On Wed, Oct 05, 2011 at 01:46AM, sebb wrote:
> On 5 October 2011 00:25, Oleg Kalnichevski <olegk@apache.org> wrote:
> > On Tue, 2011-10-04 at 15:52 -0700, Konstantin Boudnik wrote:
> >> Hello
> >>
> >> Firstly, I'd like to thank this group of people for outstanding job in
> >> development Java layer for generic HTTP: very nicely and conveniently done!
> >>
> >> Now, once I am done with being nice I have a question ;) Here's the use case
> >> where I don't see a good workaround for.
> >>
> >> We have that piece of functionality in our app. where we need to grab a little
> >> sample of a web.object (e.g. a file somewhere on a web-server side). Once the
> >> sample is obtained the software isn't interested in the rest of the content
> >> and would like to close the InputStream (actually it happens to be
> >> ContentLengthInputStream) from that object. The situation we are in is that
we
> >> can't set ContentLength on the client side and the connection is forcefully
> >> kept alive from the server side.
> >>
> >> The objects (files) are pretty large (10s or 100s of Mb) and we don't want to
> >> keep downloading the rest of the content every time we need to sample. By
> >> looking into the code of that class above I see that close() method keeps
> >> pumping the data until ContentLength is reached. This seems to be troublesome
> >> in cases when the app's code is rapidly opens a whole bunch of connections to
> >> different remote objects for the sampling purposes and then trying to close
> >> those. However, close() doesn't do any real closing of the socket input so
> >> data transfer continues and I might end up with OOME ;(
> >>
> >> Considering above restrictions is their any advisable solution for the problem
> >> I am facing? I believe patching ContentLengthInputStream might be pretty
> >> tricky because ContentLengthInputStream can't actually close SessionInputBuffer.
> >>
> >> Any hints would be highly appreciated!
> >
> > Konstantin
> >
> > Per default HttpClient always makes an attempt to keep a persistent
> > connection alive. This is the reason for #close() method of
> > ContentLengthInputStream always reading from the underlying connection
> > until the end of the message. One can use HttpUriRequest#abort() to
> > immediately shut down the underlying connection (if allocated) and
> > remove it from the connection pool.
> 
> It sounds as if the application will always want to abort the transfer
> for such requests, so would it be better not to use a persistent
> connection?

I am not sure this is possible: we are wrapping AWS Java API and this seems to
be the only kind of connections they are provide. Also, transfer termination
is just a single use case.

Thanks,
  Cos

> And would that affect how close works?
> 
> Just curious.
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
> 

Mime
View raw message