hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sam Berlin" <sber...@gmail.com>
Subject Re: Keep-Alive timeout & persistent connections
Date Mon, 30 Jun 2008 19:00:12 GMT
I can take a look later tonight -- feel free to do it sooner if you'd
like to rule it out as a cause of the leak & debug that.

On Mon, Jun 30, 2008 at 2:56 PM, Oleg Kalnichevski <olegk@apache.org> wrote:
> On Mon, 2008-06-30 at 14:54 -0400, Sam Berlin wrote:
>> Yeah, that makes sense.  Dunno what was going through my head at the
>> time, but for some reason I thought it made sense to calculate the
>> time when the headers are read.  It definitely simplifies the code to
>> do it just when the connection is released.
>>
>> Sam
>>
>
> Can you take care of that or shall I do it?
>
> Oleg
>
>
>> On Mon, Jun 30, 2008 at 2:51 PM, Oleg Kalnichevski <olegk@apache.org> wrote:
>> > On Mon, 2008-06-30 at 10:43 -0400, Sam Berlin wrote:
>> >> Yikes, that's no good.  Did you test this using the first revision of
>> >> the fix or the second (changed to use 'setIdleDuration' in
>> >> ManagedClientConnection)?
>> >>
>> >> Think it'd be possible to isolate the problem to a unit test?
>> >>
>> >> Sam
>> >>
>> >
>> > Sam,
>> >
>> > Maybe unrelated to the resource leak but something that struck me as
>> > odd. Currently AbstractClientConnAdapter starts counting the idle time
>> > from the moment of receiving the last header. This does not seem right
>> > to me. I guess we should start counting from the moment of the last I/O
>> > on that connection. For simplicity reasons I think we can assume this to
>> > be the moment the connection is released back to the manager, as
>> > connections get released automatically immediately when the data stream
>> > returns EOF.
>> >
>> > Does this make sense?
>> >
>> > Oleg
>> >
>> >
>> >> On Mon, Jun 30, 2008 at 10:33 AM, Paul Lindner <plindner@hi5.com>
wrote:
>> >> >
>> >> > I tested this code using production traffic and the results are not
so good.
>> >> >  Ended up with a connection/memory leak.  The jmap histogram of a server
in
>> >> > this bad state is attached.
>> >> >
>> >> > Also, I had some issues with one host, it used a keepalive with a timeout=1.
>> >> >  It appears that the route pool filled up with connections.  It also
appears
>> >> > that when it closes an expired free connection that it is not released
back
>> >> > into the pool...  Here's the debug output showing the final connection
>> >> > allocation and the attempt to get a connection from a full pool.  It
appears
>> >> > that Freeing the connection puts it back on the Idle pool, but it becomes
>> >> > inaccessible for some reason.
>> >> >
>> >> >
>> >> > 2008-06-29 19:22:27,761 [http-80-55] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager  -
>> >> > ThreadSafeClientConnManager.getConnection:
>> >> > HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com], timeout =
>> >> > 50002008-06-29 19:22:27,761 [http-80-55] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Getting free connection
>> >> > [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> >> > 2008-06-29 19:22:27,761 [http-80-55] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Closing expired
free
>> >> > connection [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> >> > 2008-06-29 19:22:27,761 [http-80-55] DEBUG
>> >> > org.apache.http.impl.conn.DefaultClientConnection  - Connection closed
>> >> > 2008-06-29 19:22:27,761 [http-80-55] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - No free connections
>> >> > [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> >> > 2008-06-29 19:22:27,761 [http-80-55] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Available capacity:
1
>> >> > [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> >> > 2008-06-29 19:22:27,761 [http-80-55] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Creating new connection
>> >> > [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]]
>> >> > 2008-06-29 19:22:27,761 [http-80-55] DEBUG
>> >> > org.apache.http.impl.conn.DefaultClientConnection  - Connection closed
>> >> > 2008-06-29 19:22:27,818 [http-80-55] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Freeing connection
>> >> > [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> >> > 2008-06-29 19:22:27,818 [http-80-55] DEBUG
>> >> > org.apache.http.impl.conn.IdleConnectionHandler  - Adding connection
at:
>> >> > 12147925478182008-06-29 19:22:27,818
>> >> > [http-80-55] DEBUG org.apache.http.impl.conn.tsccm.ConnPoolByRoute
 -
>> >> > Notifying no-one, there are no waiting threads
>> >> >
>> >> > -----------------------------------------------
>> >> > 2008-06-29 19:22:29,274 [http-80-26] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager  -
>> >> > ThreadSafeClientConnManager.getConnection:
>> >> > HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com], timeout =
5000
>> >> > 2008-06-29 19:22:29,274 [http-80-26] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Getting free connection
>> >> > [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> >> > 2008-06-29 19:22:29,274 [http-80-26] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Closing expired
free
>> >> > connection [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> >> > 2008-06-29 19:22:29,274 [http-80-26] DEBUG
>> >> > org.apache.http.impl.conn.DefaultClientConnection  - Connection closed
>> >> > 2008-06-29 19:22:29,274 [http-80-26] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - No free connections
>> >> > [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> >> > 2008-06-29 19:22:29,274 [http-80-26] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Available capacity:
0
>> >> > [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> >> > 2008-06-29 19:22:29,274 [http-80-26] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Need to wait for
>> >> > connection [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> >> > 2008-06-29 19:22:29,331 [http-80-79] DEBUG
>> >> > org.apache.http.impl.conn.DefaultClientConnection  - Connection shut
down
>> >> > 2008-06-29 19:22:29,331 [http-80-79] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Freeing connection
>> >> > [HttpRoute[{}->http://ads.socialmedia.com]][null]
>> >> > 2008-06-29 19:22:29,331 [http-80-79] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Notifying thread
waiting
>> >> > on any pool
>> >> > 2008-06-29 19:22:29,331 [http-80-26] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - No free connections
>> >> > [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> >> > 2008-06-29 19:22:29,331 [http-80-26] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Available capacity:
0
>> >> > [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> >> > 2008-06-29 19:22:29,331 [http-80-26] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Need to wait for
>> >> > connection [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> >> > 2008-06-29 19:22:29,331 [http-80-26] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - No free connections
>> >> > [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> >> > 2008-06-29 19:22:29,331 [http-80-26] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Available capacity:
0
>> >> > [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> >> > 2008-06-29 19:22:29,331 [http-80-26] DEBUG
>> >> > org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Need to wait for
>> >> > connection [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> >> >
>> >> >
>> >> >
>> >> > Size    Count   Class description
>> >> > -------------------------------------------------------
>> >> > 684164112       558913  byte[]
>> >> > 492584016       2134710 char[]
>> >> > 209551728       4365661 java.util.concurrent.locks.ReentrantLock$NonfairSync
>> >> > 209517696       4364952 java.util.concurrent.ConcurrentHashMap$Segment
>> >> > 144470200       4364956 java.util.concurrent.ConcurrentHashMap$HashEntry[]
>> >> > 101379880       2534497 java.lang.String
>> >> > 65693112        628849  java.lang.Object[]
>> >> > 47067200        588340  com.friend.data.UserAppBean
>> >> > 41467120        272810  java.util.concurrent.ConcurrentHashMap$Segment[]
>> >> > 37081896        272661  org.apache.catalina.session.StandardSession
>> >> > 32830608        683971  java.util.HashMap$Entry
>> >> > 30965200        275343  java.util.Hashtable$Entry[]
>> >> > 24135760        603394  java.util.ArrayList
>> >> > 22612816        115641  java.util.HashMap$Entry[]
>> >> > 19871936        620998  java.util.Date
>> >> > 19642320        272810  java.util.concurrent.ConcurrentHashMap
>> >> > 18644208        224156  int[]
>> >> > 17608256        275129  java.util.Hashtable
>> >> > 14569296        303527  java.util.concurrent.ConcurrentHashMap$HashEntry
>> >> > 13089600        272700  java.beans.PropertyChangeSupport
>> >> > 13054072        97229   * ConstMethodKlass
>> >> > 12340480        154256  org.apache.tomcat.util.buf.MessageBytes
>> >> > 11679960        97229   * MethodKlass
>> >> > 10593856        165529  org.apache.tomcat.util.buf.ByteChunk
>> >> > 10529104        10652   * ConstantPoolKlass
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > ---------------------------------------------------------------------
>> >> > To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
>> >> > For additional commands, e-mail: dev-help@hc.apache.org
>> >> >
>> >> >
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
>> >> For additional commands, e-mail: dev-help@hc.apache.org
>> >>
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
>> > For additional commands, e-mail: dev-help@hc.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
>> For additional commands, e-mail: dev-help@hc.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Mime
View raw message