hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: Strange SSL error
Date Wed, 20 May 2015 15:26:20 GMT
On Wed, 2015-05-20 at 14:53 +0000, Mark A. Claassen wrote:
> (3) or whenever message content is fully consumed.
> > response closure but at as soon as the content stream returns -1
> 
> Nice.  I never thought to check that.  Unfortunately, this wasn't happening in my test
case.  We have our own message structure where we know how long the response is going to be
and so never read past the end (getting the -1).  However, because we don't know the length
immediately, we can't put it in the content length header.  I would imagine that this doesn't
give HttpClient a chance to detect the EOF condition.
> 

This does not seem to add up. HttpClient keeps connections alive only if
it is sure the last message has been fully consumed. Otherwise it drops
the connection as non-reusable.

Oleg

> I believe I am using the library "correctly", but I guess I found a condition use case
that corrupts the pool a bit.  Is this something that can be considered for a future version,
or is this one of those things where making a slight change would cause a lot of complications?
> 
> Mark Claassen
> Senior Software Engineer
> 
> Donnell Systems, Inc.
> 130 South Main Street
> Leighton Plaza Suite 375
> South Bend, IN  46601
> E-mail: mailto:mclaassen@ocie.net
> Voice: (574)232-3784
> Fax: (574)232-4014
>   
> 
> 
> -----Original Message-----
> From: Oleg Kalnichevski [mailto:olegk@apache.org] 
> Sent: Wednesday, May 20, 2015 9:41 AM
> To: HttpClient User Discussion
> Subject: Re: Strange SSL error
> 
> On Wed, 2015-05-20 at 13:30 +0000, Mark A. Claassen wrote:
> >   > 'updated' represents the time of the last update on the client 
> > side as opposed to 'expiry' which represents keep-alive period
> > 
> > Maybe, but the expire time is then set based on the new value of "updated".  So,
even though no network activity occurred, the new expiration time is set as if there was.

> > The impact of this is that, after reading a response from the server, any processing
time that is done between the reading of the data and the releasing of the connection is ignored.

> 
> You see, the connection release is triggered whenever either of these events take place.
> 
> (1) request is aborted
> (2) response is closed
> (3) or whenever message content is fully consumed.
> 
> So, in the normal course of request execution the expiry time gets updated not at the
time of response closure but at as soon as the content stream returns -1 (end of stream).
That is, pretty much at the same time as the keep-alive timer starts ticking at the server
side.
> 
> Oleg
>  
> 
> 
> >  If the KeepAlive duration is 5 seconds, but some code takes 6 seconds to process
the results, an expired entry will be placed back into the pool.  This might not be a big
deal, but it defeats the purpose of the check.  It will also force a stale connection check
on a connection that is known to be expired.
> > 
> > This is assuming the pattern of:
> > 	Open Connection
> > 	try {
> > 		Send request
> > 		Read response
> > 		Do some initial processing of the response
> > 	}
> > 	finally {
> > 		Close connection
> > 	}
> > 
> > Perhaps the "newExpiry" should be calculated before the "updated" is set to the
current time?
> >  
> >     public synchronized void updateExpiry(final long time, final TimeUnit tunit)
{
> >         Args.notNull(tunit, "Time unit");
> >         this.updated = System.currentTimeMillis();
> >         final long newExpiry;
> >         if (time > 0) {
> >             newExpiry = this.updated + tunit.toMillis(time);
> >         } else {
> >             newExpiry = Long.MAX_VALUE;
> >         }
> >         this.expiry = Math.min(newExpiry, this.validityDeadline);
> >     }
> > 
> > 
> > 
> > Mark Claassen
> > Senior Software Engineer
> > 
> > Donnell Systems, Inc.
> > 130 South Main Street
> > Leighton Plaza Suite 375
> > South Bend, IN  46601
> > E-mail: mailto:mclaassen@ocie.net
> > Voice: (574)232-3784
> > Fax: (574)232-4014
> >   
> > -------------------------------------------
> > Confidentiality Notice: OCIESERVICE
> > -------------------------------------------
> > The contents of this e-mail message and any attachments are intended solely for
the addressee(s) named in this message. This communication is intended to be and to remain
confidential. If you are not the intended recipient of this message, or if this message has
been addressed to you in error, please immediately alert the sender by reply e-mail and then
delete this message and its attachments. Do not deliver, distribute, copy, disclose the contents
or take any action in reliance upon the information contained in the communication or any
attachments.
> > 
> > -----Original Message-----
> > From: Oleg Kalnichevski [mailto:olegk@apache.org]
> > Sent: Tuesday, May 19, 2015 4:26 AM
> > To: HttpClient User Discussion
> > Subject: Re: Strange SSL error
> > 
> > On Fri, 2015-05-15 at 16:08 +0000, Mark A. Claassen wrote:
> > > Thanks.  However, I am still getting the error; even when I do: 
> > > connectionManager.setValidateAfterInactivity(1);
> > > 
> > > I think this is just because the server can close a keepalive 
> > > connection at any time.  So, if the server decides to kill off the 
> > > connection in that time slice, the request fails.  This most often 
> > > comes back as a  org.apache.http.NoHttpResponseException which the 
> > > default retry handler will not retry.  It seems the only way I can 
> > > be sure than my connections succeed is to disable keep-alives 
> > > entirely with a ConnectionReuseStrategy.  (Also, Using
> > > RequestConfig.setStaleConnectionCheckEnabled(true) seems works 
> > > better than the setting the setValidateAfterInactivity to 1.  I am 
> > > not sure
> > > why.)
> > > 
> > > Question:
> > > releaseConnection in PoolingHttpClientConnectionManager calls updateExpiry
in PoolEntry.  However, this method in PoolEntry also sets the "updated" time.  What is "updated"
supposed to represent?  If it is mainly used to test the keepalive stuff, then it should be
updated based on network activity.  It doesn’t seem that changing a time to expire value
should be counted as "activity" on the connection.
> > > 
> > > 
> > 
> > 'updated' represents the time of the last update on the client side as opposed to
'expiry' which represents keep-alive period as communicated by the server. One might want
to close connections after, say, 3 seconds of inactivity on client side even though the server's
keep-alive is, say, 5 seconds.
> > 
> > Oleg
> > 
> > > 
> > > Mark Claassen
> > > Senior Software Engineer
> > > 
> > > Donnell Systems, Inc.
> > > 130 South Main Street
> > > Leighton Plaza Suite 375
> > > South Bend, IN  46601
> > > E-mail: mailto:mclaassen@ocie.net
> > > Voice: (574)232-3784
> > > Fax: (574)232-4014
> > >   
> > > -------------------------------------------
> > > Confidentiality Notice: OCIESERVICE
> > > -------------------------------------------
> > > The contents of this e-mail message and any attachments are intended solely
for the addressee(s) named in this message. This communication is intended to be and to remain
confidential. If you are not the intended recipient of this message, or if this message has
been addressed to you in error, please immediately alert the sender by reply e-mail and then
delete this message and its attachments. Do not deliver, distribute, copy, disclose the contents
or take any action in reliance upon the information contained in the communication or any
attachments.
> > > 
> > > 
> > > -----Original Message-----
> > > From: Oleg Kalnichevski [mailto:olegk@apache.org]
> > > Sent: Thursday, May 14, 2015 4:38 AM
> > > To: HttpClient User Discussion
> > > Subject: Re: Strange SSL error
> > > 
> > > On Wed, 2015-05-13 at 20:58 +0000, Mark A. Claassen wrote:
> > > > The 4.4.1 code doesn't seem to help.
> > > > 
> > > > I have been able to reproduce the issue more regularly now.  It seems
to have to do with keep-alives and if the client takes longer to read the message than the
keep alive value.
> > > > 
> > > 
> > > Hi Mark
> > > 
> > > Then, it is all very simple. There are several ways to make HttpClient either
discard potentially half-closed connections, test them for 'staleness' or automatically recover
from NoHttpResponseException.
> > > 
> > > 
> > > > In my test case I open the connection, read all the data, sleep for a
while, then close the connection.
> > > > If my sleep is a bit longer than the keep-alive value, I will get a org.apache.http.NoHttpResponseException.
 If my sleep value is larger, I will get a java.net.SocketException.
> > > > (Keep-Alive is 5000 millis.  If I sleep for 6000, I will get a NoHttpResponseException.
 If I sleep for 11000, I will get a SocketException.
> > > > 
> > > 
> > > The default maximum inactivity period used by the pooling connection 
> > > manager is exactly 5000 ms. Please try reducing this value to, say,
> > > 2000 ms
> > > 
> > > ---
> > > PoolingHttpClientConnectionManager cm = new 
> > > PoolingHttpClientConnectionManager();
> > > cm.setValidateAfterInactivity(2000);
> > > 
> > > CloseableHttpClient client = HttpClients.custom()
> > >         .setConnectionManager(cm)
> > >         .build();
> > > ---
> > > 
> > > This should make the problem go away.
> > > 
> > > > If I use the NoConnectionReuseStrategy, the problem goes away.
> > > > 
> > > > Is something set up for my keep alives?  I put some breakpoints in CPool.java.
 I can see connections being created, but the validate() method of CPool is never getting
called.
> > > > 
> > > > I am curious this part of AbstractConnPool.java.  This seems like if the
server invalidated a connection early, the validate check would never happen.
> > > > 
> > > >                     } else if (this.validateAfterInactivity > 0) {
> > > >                         if (entry.getUpdated() + this.validateAfterInactivity
<= System.currentTimeMillis()) {
> > > >                             if (!validate(entry)) {
> > > >                                 entry.close();
> > > >                             }
> > > >                         }
> > > >                     }
> > > > Scenario:
> > > > 	Server sends data.
> > > > 	Client reads packet, processes it for 6 seconds
> > > > 	Server senses that inactivity for 5 seconds, closes connection
> > > > 	Client closes connection and places entry back in the pool
> > > > 	Connection immediately leased to another thread
> > > > 	Time between release and close is almost nothing
> > > > 	Pool releases stale connection.
> > > > 
> > > 
> > > You can force TTL (total time to live) for connections to avoid this 
> > > problem
> > > 
> > > ---
> > > PoolingHttpClientConnectionManager cm = new 
> > > PoolingHttpClientConnectionManager(3000, TimeUnit.MILLISECONDS); 
> > > cm.setValidateAfterInactivity(1000);
> > > 
> > > CloseableHttpClient client = HttpClients.custom()
> > >         .setConnectionManager(cm)
> > >         .build();
> > > ---
> > > 
> > > Hope this helps
> > > 
> > > Oleg
> > > 
> > > 
> > > --------------------------------------------------------------------
> > > - To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> > > 
> > > 
> > > --------------------------------------------------------------------
> > > - To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> > > 
> > 
> > 
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> > 
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> > 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> 



---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message