jakarta-jcs-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Smuts <aasm...@wisc.edu>
Subject RE: 1.2.4-dev fix -- RE: Excessive number of JCS objects in memory
Date Tue, 01 Feb 2005 05:29:04 GMT
I'm running with commons collections 3.0 with jcs 1.2.4-dev.  The
collections LRUMap is used for disk key storage and purgatory.  We have
our own LRU for the memory cache, which performs better.

I'll add the stats data to the admin jsp soon.  

I'm also trying to beat the hell out of the tcp lateral cache.  It's
damn fast and seems to recover nicely when a node goes down. 

It is very difficult to unit test the remote and lateral auxiliaries
since the cache only works as a singleton.  Any suggestions are welcome.

Aaron

> -----Original Message-----
> From: Aaron Smuts [mailto:aasmuts@wisc.edu]
> Sent: Monday, January 31, 2005 9:04 PM
> To: 'Turbine JCS Users List'
> Subject: 1.2.4-dev fix -- RE: Excessive number of JCS objects in
memory
> 
> I'm pretty sure that there is no memory leak with the disk cache in
> version 1.2.4-dev.  I recommend upgrading to this version.  The
release
> is in the tempbuild folder and the repository is tagged.
> 
> http://cvs.apache.org/viewcvs.cgi/jakarta-turbine-jcs/tempbuild/
> 
> I've been running in pure disk mode (max memory size = 0) for some
time
> and with a limit placed on the number of disk cache keys, I can keep
the
> memory steady.  It increased 5k after 1 million operations (a combo of
> puts, gets, and removes) but I think this is due to additional stats
> data.
> 
> I added a gc call in the manual tester program that works nicely.
Here
> is the stats and gc info after one test:
> 
> 
> Called system.gc()
> enter command:
> stats
> Region Name = testCache1
> HitCountRam = 0
> HitCountAux = 366613
> ---------------------------LRU Memory Cache
> List Size = 0
> Map Size = 0
> Put Count = 315742
> Hit Count = 0
> Miss Count = 590662
> ---------------------------Indexed Disk Cache
> Is Alive = true
> Key Map Size = 10000
> Data File Length = 13515586
> Optimize Opertaion Count = 249597
> Times Optimized = 0
> Recycle Count = 146319
> Purgatory Hits = 0
> Purgatory Size = 0
> Working = true
> Alive = false
> Empty = true
> Size = 0
> enter command:
> gc
> [Full GC 2019K->2006K(91520K), 0.1179804 secs]
> [Full GC 2006K->2006K(91520K), 0.1187335 secs]
> Called system.gc()
> enter command:
> gc
> [Full GC 2006K->2006K(91520K), 0.1178323 secs]
> [Full GC 2006K->2006K(91520K), 0.1176138 secs]
> Called system.gc()
> 
> 
> 
> My disk cache configuration is here:
> 
> # Primary Disk Cache-- faster than the rest because of memory key
> storage
>
jcs.auxiliary.DC=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheF
> actory
>
jcs.auxiliary.DC.attributes=org.apache.jcs.auxiliary.disk.indexed.Indexe
> dDiskCacheAttributes
> jcs.auxiliary.DC.attributes.DiskPath=target/test-sandbox/raf
> jcs.auxiliary.DC.attributes.MaxPurgatorySize=10000
> jcs.auxiliary.DC.attributes.MaxKeySize=10000
> jcs.auxiliary.DC.attributes.MaxRecycleBinSize=5000
> jcs.auxiliary.DC.attributes.OptimizeAtRemoveCount=300000
> 
> I never optimized, in my test.  I suggest setting
OptimizeAtRemoveCount
> to -1 for now.
> 
> My region config is here:
> 
> ##############################################################
> ################## CACHE REGIONS AVAILABLE ###################
> # Regions preconfirgured for caching
> jcs.region.testCache1=DC
>
jcs.region.testCache1.cacheattributes=org.apache.jcs.engine.CompositeCac
> heAttributes
> jcs.region.testCache1.cacheattributes.MaxObjects=0
>
jcs.region.testCache1.cacheattributes.MemoryCacheName=org.apache.jcs.eng
> ine.memory.lru.LRUMemoryCache
> jcs.region.testCache1.cacheattributes.UseMemoryShrinker=true
> jcs.region.testCache1.cacheattributes.ShrinkerIntervalSeconds=30
> jcs.region.testCache1.cacheattributes.MaxMemoryIdleTimeSeconds=300
> jcs.region.testCache1.cacheattributes.MaxSpoolPerRun=100
>
jcs.region.testCache1.elementattributes=org.apache.jcs.engine.ElementAtt
> ributes
> jcs.region.testCache1.elementattributes.IsEternal=false
> jcs.region.testCache1.elementattributes.MaxLifeSeconds=60000
> jcs.region.testCache1.elementattributes.IsLateral=true
> jcs.region.testCache1.elementattributes.IsRemote=true
> 
> > -----Original Message-----
> > From: Wyatt, Allen [mailto:Allen.Wyatt@travelocity.com]
> > Sent: Monday, January 31, 2005 4:39 PM
> > To: Turbine JCS Users List
> > Subject: RE: Excessive number of JCS objects in memory
> >
> > Regarding using newer JCS - which one?  What is the latest "stable"
> > version - is it "jcs-1.2.3-dev.jar" (the "-dev" in the name makes me
> > think not).
> >
> > Right now we're using JCS 1.1 (downloaded Oct 19 2004) and commons
> > collections 2.1.
> >
> > Putting heavy load on the disk cache (each of the 100s of user
> sessions
> > to each webserver on our website has the potential to put something
in
> > the disk cache as well as other parts of the application putting
stuff
> > into it).
> >
> > Thanks for all the help.
> >
> > -----Original Message-----
> > From: Smuts, Aaron [mailto:aaronsm@amazon.com]
> > Sent: Monday, January 31, 2005 5:52 PM
> > To: Turbine JCS Users List
> > Subject: RE: Excessive number of JCS objects in memory
> >
> > The abstract disk cache still uses that old write lock for each item
> put
> > into the disk cache.  The indexed disk cache used to use it.  The
Core
> > disk cache class now uses the Doug Lea read write lock classes.
> >
> > I will convert the abstact disk cache to use the Doug Lea read write
> > lock also.
> >
> > What version of JCS are you using?  You should probably upgrade JCS
> and
> > the commons collections libraries.  Your problem earlier looked like
a
> > commons issue.
> >
> > What kind of load are you putting on the disk cache?
> >
> > Aaron
> >
> > -----Original Message-----
> > From: Wyatt, Allen [mailto:Allen.Wyatt@travelocity.com]
> > Sent: Monday, January 31, 2005 3:05 PM
> > To: turbine-jcs-user@jakarta.apache.org
> > Subject: Excessive number of JCS objects in memory
> >
> > We're having a problem with a slow memory leak in production.  We
took
> a
> > heapdump and saw that there were 337,999
> > org.apache.jcs.utils.locking.ReadWriteLock objects in memory and
> 337,967
> > org.apache.jcs.utils.locking.RwLockHolder objects in memory.  These
> > objects are taking about 21.5 megabytes of memory altogether.  Any
> > reason for having this many of these objects?  What might cause
there
> to
> > be an excessive number of these objects?
> >
> >
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> turbine-jcs-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > turbine-jcs-user-help@jakarta.apache.org
> >
> >
> >
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> turbine-jcs-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > turbine-jcs-user-help@jakarta.apache.org
> >
> >
> >
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> turbine-jcs-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> turbine-jcs-user-help@jakarta.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
turbine-jcs-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
turbine-jcs-user-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: turbine-jcs-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: turbine-jcs-user-help@jakarta.apache.org


Mime
View raw message