hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sadeep Jayasumana <gayansad...@gmail.com>
Subject Re: HTTPCore-NIO: Server rejecting socket connections?
Date Tue, 31 Jan 2012 16:12:29 GMT
Hi Oleg,

On Tue, Jan 31, 2012 at 7:48 PM, Oleg Kalnichevski <olegk@apache.org> wrote:

> On Mon, 2012-01-30 at 14:56 +0100, Oleg Kalnichevski wrote:
> > On Mon, 2012-01-30 at 17:45 +0530, Sadeep Jayasumana wrote:
> > > Hi,
> > >
> > > On Mon, Jan 30, 2012 at 5:15 PM, Oleg Kalnichevski <olegk@apache.org>
> wrote:
> > >
> > > > On Sat, 2012-01-28 at 23:46 +0530, Sadeep Jayasumana wrote:
> > > > > Hi,
> > > > >
> > > > > On Fri, Jan 27, 2012 at 6:50 PM, Oleg Kalnichevski <
> olegk@apache.org>
> > > > wrote:
> > > > >
> > > > > > On Fri, 2012-01-27 at 15:18 +0530, Sadeep Jayasumana wrote:
> > > > > > > Hi,
> > > > > > >
> > > > > > > I have written an HTTP server using HTTPCore-NIO and currently
> > > > testing it
> > > > > > > with httpcore-ab [1]. I have been seeing very good TPS
values
> so
> > > > far. But
> > > > > > > when I try to send 5KB HTTP messages with 2500 and concurrency
> > > > level, 50
> > > > > > > messages per thread, I'm getting the following exception
> several
> > > > times
> > > > > > from
> > > > > > > httpcore-ab and it results in "Failed requests" in the
> httpcore-ab
> > > > > > output.
> > > > > > > No exceptions are displayed from the server side. Hence,
I
> think the
> > > > > > server
> > > > > > > is rejecting some socket connections because it's already
fully
> > > > loaded.
> > > > > > >
> > > > > > > I'm running the server and load generator on two server
> machines
> > > > that are
> > > > > > > connected with Gigabit Ethernet. Each server has 16 CPU
cores,
> 16 GB
> > > > > > memory
> > > > > > > etc.
> > > > > > >
> > > > > > > I have increased the socket timeout in both httpcore-ab
and
> server
> > > > side
> > > > > > > ListeningIOReactor to 360000. 16 threads are allocated
for the
> > > > reactor.
> > > > > > > Messages received by the server are processed in a separate
> worker
> > > > pool
> > > > > > > which has ~500 threads. I have also increased the open
file
> limit of
> > > > my
> > > > > > OS
> > > > > > > (Debian) to 65535.
> > > > > > >
> > > > > > > Is there any tuning parameter of HTTPCore that I can used
to
> get rid
> > > > of
> > > > > > > this exception? Perhaps, something similar to
> "acceptQueueSize" in
> > > > Jetty
> > > > > > >  [2]?
> > > > > > >
> > > > > >
> > > > > > Hi Sadeep
> > > > > >
> > > > > > A couple of points in no particular logical order.
> > > > > >
> > > > > > (1) HttpCore does not depend on a logging toolkit and does not
> do any
> > > > > > logging per default. If it ever encounters an abnormal situation,
> > > > > > instead of logging such an event it throws an exception and
> terminates.
> > > > > > Quite often this is not desirable, so one can provide an
> exception
> > > > > > handler that can log using a a logging toolkit of choice or
> handle a
> > > > > > particular type of exception in an application specific way.
> > > > > >
> > > > > > If the server did not terminate under with an exception, I kind
> of
> > > > > > suspect it was still trying to chew through the load without
> > > > > > encountering anything abnormal. Unless you have added a custom
> > > > exception
> > > > > > handler.
> > > > > >
> > > > > > (2) HttpCore does not have a concept of accepted connection
> queue. As
> > > > > > soon as the listener accepts an incoming connection its channel
> gets
> > > > > > immediately added to the underlying i/o selector. No intermediate
> > > > > > queuing takes place.
> > > > >
> > > > >
> > > > > > (3) Given that the connection times out on the client, this
> sounds more
> > > > > > like a client side issue. Was connection really suck in
> > > > > > Socket#socketConnect for 360000?
> > > > > >
> > > > >
> > > > > I'm using httpcore-ab [1] as the client. I have set socket timeout
> of
> > > > using
> > > > > -t 360000, and changed the connection timeout with the patch shown
> below.
> > > > > But I'm starting to get this error well before 360000 millis past
> the
> > > > test
> > > > > startup. Therefore, I there is no possibility that the connection
> attempt
> > > > > hangs for 360000 millis before printing this exception. I was
> thinking
> > > > > whether there are upper bounds for these limits that cause my
> settings to
> > > > > be neglected.
> > > > >
> > > >
> > > > You see. If java.net.ConnectException: Connection timed out is
> thrown by
> > > > java.net.PlainSocketImpl.socketConnect before 360000 millis elapse,
> > > > there must be something wrong with the client side.
> > > >
> > >
> > > Could this be a possible bug in httpcore-ab code? I'm quite sure that
> it
> > > doesn't wait 360000 millis before throwing this exception.
> > >
> >
>
> Hi Sadeep
>
> It did turn out to be an issue with the way HttpCore AB initialized
> connections. It was basically using an old (Java 1.0) socket constructor
> whose behavior can differ between OS platforms and possibly between JRE
> major releases with regards to connect timeout handling. The connect
> timeout value set in the HttpParams simply had no effect and therefore
> connections in your tests were timing out before 360000 milliseconds
> elapsed (probably using some sort of a system default).
>
> I committed a fix for the problem
>
> http://svn.apache.org/viewvc?rev=1238566&view=rev
>
> Please review
>

Many thanks for looking into this. I will review this at my earliest and
let you know the outcome.

Thanks,
Sadeep


>
> Oleg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
>


-- 

Sadeep Jayasumana****

Email: gayansadeep@gmail.com****

Phone: +94-77-2266507

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message