Mailing-List: contact httpcomponents-dev-help@jakarta.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "HttpComponents Project" <httpcomponents-dev@jakarta.apache.org>
Received-SPF: pass (herse.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=X-YMail-OSG:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID;
  b=rPOfsJUIvCECo/2JQ888Oq9z2LfXYxNt22WaopHisu95bM6mxOzA7z9G4IORzlOIYzbx8sh42IaFxLk88tigt+aL6expYBn7Ufrt+XXoOiKl9QJOBzr0Skaql7weu/gwZGrmWKU7pXLAVyfxr+1CTalgYCfMpJid1SOv68+ZsFI=;
Date: Fri, 23 Feb 2007 11:47:40 -0800 (PST)
From: Tatu Saloranta <cowtowncoder@yahoo.com>
Subject: Re: Performance of stale connection check
To: HttpComponents Project <httpcomponents-dev@jakarta.apache.org>
In-Reply-To: <1172141926.4760.50.camel@okhost>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit
Message-ID: <486128.20035.qm@web32805.mail.mud.yahoo.com>


--- Oleg Kalnichevski <olegk@apache.org> wrote:

...
> > This may be too late for HttpClient 3.x, but if
not,
> > would it be easy to allow pluggability of the
staleness
> > checks (I have not checked the code -- if it
...
> Hi Tatu,
> 
> No, unfortunately it would not. HttpConnection in
> HttpClient 3.x is a
> concrete class and a pretty messy one on top of
> that. It is very
> difficult to customize its behavior without forking
> the whole damn
> thing.   

Ok. I thought this might be the case, just wanted to
verify. I don't mind waiting for 4.0, since it seems
very promising based on discussions and progress
so far.

> Things will be radically better with HttpClient 4.0.
> Some time ago I put
> HttpCore protocol layer on top of MINA transport in
> less than half a day
> of coding. So, one can provide an alternative
> implementation of
> HttpConnection interfaces and implement some custom
> logic in place of
> the default stale connection check. 

That would be good.

> >  For many users,
> > Jdk 1.4. requirement would be quite acceptable,
> > and perhaps such a plug-in could be made
> accessible
> > as a separate contribution. So it would not be
part
> > of the core, but available for those who need it.
> > 
> 
> This is certainly an option. I experimented with
> mixed classic IO / NIO
> connection implementations a while ago at the very
> early days of
> HttpCore. I observed a noticeable performance
> degradation when using
> SocketChannel backed connections compared to those
> backed a Socket
> without an associated Channel. So, I am not sure it
> is worth paying such
> a price. 

Actually, I may have misunderstood things here. My
initial
thought was that it would be possible to access the
underlying channel for a Socket, even in blocking
mode;
and to be able to do toggle handling of some
operations
(kind of enabling non-blocking mode just for staleness
checks). But looking at javadocs, I suspect I was
wrong there.

So I wasn't suggesting mandating use of NIO for other
operations at all. I know your reservations regarding
performance, and by now think NIO happens to fall into
wrong
side of 80/20 rule in many/most common use cases.

...
> > one of the things that can noticeably increase
> > latency;
> > being especially problematic with services whose
> SLA
> > is, say, 10 ms, at 99% fractile).
> 
> I find this a little extreme. My recommendation was

Which part? SLAs in question, or approach?

I guess what I'm just saying is that for some use
cases
(high-speed LAN use case, where the whole
request/reply
interchange can be done in a msec or two) 20 msec
delay
is unacceptable, whereas for many others it is not.

So if there was a way to resolve this, without having
to use NIO itself, it'd be great. But I guess not.

Your points wrt. staleness checks feasibility in
general
are valid ones: I didn't realize there's lots of
scepticism towards usefulness of staleness checks
(esp.
as implements).

> always to disable
> the stale connection check and to provide a robust
> request retry handler
> instead. A production quality application must be
> prepared to deal with
> and gracefully recover from all sort of I/O failures
> _anyways_, so what

Yes... it's just that connection reuse and pooling is
well abstracted behind http components, and as such
it is hard to tell "real" problems (service going
down)
from business as usual (server closing persistent
connections to conserve resources).

And disabling the whole connection pooling (and
persistent
connections) would indeed be a bit extreme.

> is the point in treating I/O failures due to stale
> connections somehow
> differently?

I think the other reply summed it up well -- it is
question of responsibility wrt. who is to handle
which kinds of failures. Staleness check is not
visibile to the client, so it shouldn't have to
deal with it.

But refactorings done for 4.0 should allow more
modular
approach to connection pool handling, which is a
very good thing.

Once again, thank you for your thoughtly explanations,

-+ Tatu +-


____________________________________________________________________________________
8:00? 8:25? 8:40? Find a flick in no time 
with the Yahoo! Search movie showtime shortcut.
http://tools.search.yahoo.com/shortcuts/#news

---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org