hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: HTTPCore-NIO: Server rejecting socket connections?
Date Fri, 27 Jan 2012 13:20:06 GMT
On Fri, 2012-01-27 at 15:18 +0530, Sadeep Jayasumana wrote:
> Hi,
> 
> I have written an HTTP server using HTTPCore-NIO and currently testing it
> with httpcore-ab [1]. I have been seeing very good TPS values so far. But
> when I try to send 5KB HTTP messages with 2500 and concurrency level, 50
> messages per thread, I'm getting the following exception several times from
> httpcore-ab and it results in "Failed requests" in the httpcore-ab output.
> No exceptions are displayed from the server side. Hence, I think the server
> is rejecting some socket connections because it's already fully loaded.
> 
> I'm running the server and load generator on two server machines that are
> connected with Gigabit Ethernet. Each server has 16 CPU cores, 16 GB memory
> etc.
> 
> I have increased the socket timeout in both httpcore-ab and server side
> ListeningIOReactor to 360000. 16 threads are allocated for the reactor.
> Messages received by the server are processed in a separate worker pool
> which has ~500 threads. I have also increased the open file limit of my OS
> (Debian) to 65535.
> 
> Is there any tuning parameter of HTTPCore that I can used to get rid of
> this exception? Perhaps, something similar to "acceptQueueSize" in Jetty
>  [2]?
> 

Hi Sadeep

A couple of points in no particular logical order.

(1) HttpCore does not depend on a logging toolkit and does not do any
logging per default. If it ever encounters an abnormal situation,
instead of logging such an event it throws an exception and terminates.
Quite often this is not desirable, so one can provide an exception
handler that can log using a a logging toolkit of choice or handle a
particular type of exception in an application specific way. 

If the server did not terminate under with an exception, I kind of
suspect it was still trying to chew through the load without
encountering anything abnormal. Unless you have added a custom exception
handler.

(2) HttpCore does not have a concept of accepted connection queue. As
soon as the listener accepts an incoming connection its channel gets
immediately added to the underlying i/o selector. No intermediate
queuing takes place.

(3) Given that the connection times out on the client, this sounds more
like a client side issue. Was connection really suck in
Socket#socketConnect for 360000?

(4) Quite often it is useful to see what it really going on on the I/O
reactor level. You might want to add I/O event and wire logging by
copying logging components from the contrib package [1]

I hope this helps somewhat. In any way I can say I am quite sure
HttpCore does not reject incoming connections no matter what.

Oleg

[1]
https://svn.apache.org/repos/asf/httpcomponents/httpcore/branches/4.1.x/httpcore-contrib/src/main/java/org/apache/http/contrib/logging/

> [1]
> https://svn.apache.org/repos/asf/httpcomponents/httpcore/tags/4.1.3/httpcore-ab
> [2] http://docs.codehaus.org/display/JETTY/Configuring+Connectors
> [3]
> java.net.ConnectException: Connection timed out
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
> at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
> at java.net.Socket.connect(Socket.java:529)
> at java.net.Socket.connect(Socket.java:478)
> at java.net.Socket.<init>(Socket.java:375)
> at java.net.Socket.<init>(Socket.java:189)
> at org.apache.http.benchmark.BenchmarkWorker.run(BenchmarkWorker.java:207)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> java.net.ConnectException: Connection timed out
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
> at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
> at java.net.Socket.connect(Socket.java:529)
> at java.net.Socket.connect(Socket.java:478)
> at java.net.Socket.<init>(Socket.java:375)
> at java.net.Socket.<init>(Socket.java:189)
> at org.apache.http.benchmark.BenchmarkWorker.run(BenchmarkWorker.java:207)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> 
> Thanks & Regards,



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Mime
View raw message