hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guillaume Cottenceau ...@mnc.ch>
Subject Re: threads problem with many connections
Date Fri, 01 Oct 2004 12:34:23 GMT

Thanks for your answer.

> > It seems that java.nio is capable of using only one thread for
> > several lowlevel (OS) socket connections, and is actually also
> > quite efficient.
> Guillaume, 
> Please correct me if my understanding of the problem is incorrect, it is
> Tomcat that runs out of connections, not HttpClient. In fact, there's
> only one case when HttpClient spawns a new thread, and if so is desired,
> this can be disabled. The thread management is entirely responsibility
> of your application. One does not have to use one thread per socket
> model even without NIO.

Yes, you're right. Our application is running out of threads,
e.g. Tomcat actually, not HttpClient. Sorry for not being clear
about that.

My understanding of java.nio with multiple IO connections managed
by a single java Thread (this is probable what you call
multiplexing in your previous message?), is that we could imagine
using an asynchronous executeMethod-like that would initiate a
connection in a unique httpclient Thread dealing with all the
connections, returning back immediately, and later a callback be
invoked when data is available (in httpclient situation, the
callback at the "user" level (the program using httpclient) could
be called upon completion of request).

This way, one tomcat thread could be performing several HTTP
posts itself.

In real-life, it doesn't solve all of our problems because we
have some HTTP posts that need to be executed after completion of

> > My question is, since you don't want to lose 1.2 compatibility
> > before 4.0, is there then a way to solve a typical "too many
> > threads" problem such as the one we have? Do you people never had
> > the same problem? Or have found a way to solve it?
> Feasible approach is to have one monitor thread checking on the status
> of active connections or/and processing incoming connections, and a
> number of worker threads in a shared pool to do the actual work.

Actually since probably most people using httpclient with many
connections would need that, would it make sense to have it
shared somewhere, for example in httpclient?

> > It seems the HTTP protocol doesn't have anything resembling a
> > "global timeout" for a given connection (e.g. after x seconds,
> > close the receiving channel even if server hasn't finished
> > sending), and thus normally httpclient doesn't provide such a
> > thing. Do you think this should be investigated/implemented in
> > some way?
> I do not recall HTTP protocol as such dealing with timeouts in any way.
> I may be wrong, though. This is certainly not a protocol concern.
> HttpClient does provide so called socket timeout. When set to a positive
> number, the connection will terminate with an IOException after the
> specified period of inactivity:
> http://jakarta.apache.org/commons/httpclient/apidocs/org/apache/commons/httpclient/HttpClient.html#setTimeout(int)

Yes, we're already using that (and connection timeouts as well).
We're just in trouble when the post never finishes, but actually
the server is very slow (if it sends one byte per second for
example, but on a regular basis) or goes mad.

Guillaume Cottenceau

To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org

View raw message