hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Lindner <plind...@hi5.com>
Subject Re: Keep-Alive timeout & persistent connections
Date Tue, 01 Jul 2008 23:16:21 GMT
That did it.  I'm running svn head on one production server and seeing  
very good behavior.

On Jun 30, 2008, at 6:53 PM, Sam Berlin wrote:

> Hi Paul,
>
> I think I've fixed this one -- the problem was using deleteEntry
> instead of dropEntry.  The former required the entry to be considered
> 'free', but the entry had just been allocated and was definitely not
> 'free'.  I'm pretty certain this will indirectly fix the
> connection/memory leak -- a side effect of the problem was connection
> requests would just pile up and never be serviced.
>
> Sam
>
> On Mon, Jun 30, 2008 at 10:33 AM, Paul Lindner <plindner@hi5.com>  
> wrote:
>>
>> I tested this code using production traffic and the results are not  
>> so good.
>> Ended up with a connection/memory leak.  The jmap histogram of a  
>> server in
>> this bad state is attached.
>>
>> Also, I had some issues with one host, it used a keepalive with a  
>> timeout=1.
>> It appears that the route pool filled up with connections.  It also  
>> appears
>> that when it closes an expired free connection that it is not  
>> released back
>> into the pool...  Here's the debug output showing the final  
>> connection
>> allocation and the attempt to get a connection from a full pool.   
>> It appears
>> that Freeing the connection puts it back on the Idle pool, but it  
>> becomes
>> inaccessible for some reason.
>>
>>
>> 2008-06-29 19:22:27,761 [http-80-55] DEBUG
>> org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager  -
>> ThreadSafeClientConnManager.getConnection:
>> HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com], timeout =
>> 50002008-06-29 19:22:27,761 [http-80-55] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Getting free  
>> connection
>> [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> 2008-06-29 19:22:27,761 [http-80-55] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Closing expired  
>> free
>> connection [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]] 
>> [null]
>> 2008-06-29 19:22:27,761 [http-80-55] DEBUG
>> org.apache.http.impl.conn.DefaultClientConnection  - Connection  
>> closed
>> 2008-06-29 19:22:27,761 [http-80-55] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - No free  
>> connections
>> [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> 2008-06-29 19:22:27,761 [http-80-55] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Available  
>> capacity: 1
>> [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> 2008-06-29 19:22:27,761 [http-80-55] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Creating new  
>> connection
>> [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]]
>> 2008-06-29 19:22:27,761 [http-80-55] DEBUG
>> org.apache.http.impl.conn.DefaultClientConnection  - Connection  
>> closed
>> 2008-06-29 19:22:27,818 [http-80-55] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Freeing connection
>> [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> 2008-06-29 19:22:27,818 [http-80-55] DEBUG
>> org.apache.http.impl.conn.IdleConnectionHandler  - Adding  
>> connection at:
>> 12147925478182008-06-29 19:22:27,818
>> [http-80-55] DEBUG org.apache.http.impl.conn.tsccm.ConnPoolByRoute  -
>> Notifying no-one, there are no waiting threads
>>
>> -----------------------------------------------
>> 2008-06-29 19:22:29,274 [http-80-26] DEBUG
>> org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager  -
>> ThreadSafeClientConnManager.getConnection:
>> HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com], timeout = 5000
>> 2008-06-29 19:22:29,274 [http-80-26] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Getting free  
>> connection
>> [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> 2008-06-29 19:22:29,274 [http-80-26] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Closing expired  
>> free
>> connection [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]] 
>> [null]
>> 2008-06-29 19:22:29,274 [http-80-26] DEBUG
>> org.apache.http.impl.conn.DefaultClientConnection  - Connection  
>> closed
>> 2008-06-29 19:22:29,274 [http-80-26] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - No free  
>> connections
>> [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> 2008-06-29 19:22:29,274 [http-80-26] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Available  
>> capacity: 0
>> [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> 2008-06-29 19:22:29,274 [http-80-26] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Need to wait for
>> connection [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]] 
>> [null]
>> 2008-06-29 19:22:29,331 [http-80-79] DEBUG
>> org.apache.http.impl.conn.DefaultClientConnection  - Connection  
>> shut down
>> 2008-06-29 19:22:29,331 [http-80-79] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Freeing connection
>> [HttpRoute[{}->http://ads.socialmedia.com]][null]
>> 2008-06-29 19:22:29,331 [http-80-79] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Notifying thread  
>> waiting
>> on any pool
>> 2008-06-29 19:22:29,331 [http-80-26] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - No free  
>> connections
>> [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> 2008-06-29 19:22:29,331 [http-80-26] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Available  
>> capacity: 0
>> [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> 2008-06-29 19:22:29,331 [http-80-26] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Need to wait for
>> connection [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]] 
>> [null]
>> 2008-06-29 19:22:29,331 [http-80-26] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - No free  
>> connections
>> [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> 2008-06-29 19:22:29,331 [http-80-26] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Available  
>> capacity: 0
>> [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
>> 2008-06-29 19:22:29,331 [http-80-26] DEBUG
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Need to wait for
>> connection [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]] 
>> [null]
>>
>>
>>
>> Size    Count   Class description
>> -------------------------------------------------------
>> 684164112       558913  byte[]
>> 492584016       2134710 char[]
>> 209551728       4365661 java.util.concurrent.locks.ReentrantLock 
>> $NonfairSync
>> 209517696       4364952 java.util.concurrent.ConcurrentHashMap 
>> $Segment
>> 144470200       4364956 java.util.concurrent.ConcurrentHashMap 
>> $HashEntry[]
>> 101379880       2534497 java.lang.String
>> 65693112        628849  java.lang.Object[]
>> 47067200        588340  com.friend.data.UserAppBean
>> 41467120        272810  java.util.concurrent.ConcurrentHashMap 
>> $Segment[]
>> 37081896        272661  org.apache.catalina.session.StandardSession
>> 32830608        683971  java.util.HashMap$Entry
>> 30965200        275343  java.util.Hashtable$Entry[]
>> 24135760        603394  java.util.ArrayList
>> 22612816        115641  java.util.HashMap$Entry[]
>> 19871936        620998  java.util.Date
>> 19642320        272810  java.util.concurrent.ConcurrentHashMap
>> 18644208        224156  int[]
>> 17608256        275129  java.util.Hashtable
>> 14569296        303527  java.util.concurrent.ConcurrentHashMap 
>> $HashEntry
>> 13089600        272700  java.beans.PropertyChangeSupport
>> 13054072        97229   * ConstMethodKlass
>> 12340480        154256  org.apache.tomcat.util.buf.MessageBytes
>> 11679960        97229   * MethodKlass
>> 10593856        165529  org.apache.tomcat.util.buf.ByteChunk
>> 10529104        10652   * ConstantPoolKlass
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
>> For additional commands, e-mail: dev-help@hc.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>

Paul Lindner
plindner@hi5.com




---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Mime
View raw message