jakarta-jcs-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen Wallace" <OWall...@euro.banta.com>
Subject RE: Memory Issues with JCS [java.lang.OutOfMemoryError: Java heap space]
Date Fri, 28 Oct 2005 14:10:22 GMT
Aaron,

we are replacing a hash map with JCS, because we are moving to multiple
servers and the test works fine on the map.

Now for the good news

As you suggested I ran this same test with the JGROUP section of
cache.ccf commented out and it passed.

I will next try it with Lateral TCP and UPD discovery. [Probably
Tuesday]

Thanks for you assistance.

Owen


-----Original Message-----
From: Smuts, Aaron [mailto:Aaron.Smuts@travelocity.com]
Sent: 27 October 2005 16:43
To: JCS Users List
Subject: RE: Memory Issues with JCS [java.lang.OutOfMemoryError: Java
heap space]


I'm not sure what your test proves.  It doesn't show that there is a
memory leak, only that you can consume memory faster than it can be
reclaimed.  Perhaps you'd get the same error if you used a hastable.

You have your cache setup so that every region can have 10000 items!
How may regions do you have?  Maybe you are simply putting too much in
the cache.  Why don't you think this is the problem?

(Also, I'm interested in seeing the same test without the JGroups stuff.

I don't recommend using JGroups at all.  It's not even in the repository
any more.  The lateral TCP with UDP discovery is much better.)

Aaron

> -----Original Message-----
> From: Owen Wallace [mailto:OWallace@euro.banta.com]
> Sent: Thursday, October 27, 2005 11:09 AM
> To: jcs-users@jakarta.apache.org
> Subject: Memory Issues with JCS [java.lang.OutOfMemoryError: Java heap
> space]
> 
> Hi,
> 
> I'm having an issue with the my heap space. it looks like a memory
leak,
> 
> We testing our app which uses JCS as it caching mechanism and it was
fine,
> how ever after about 10 hours running in production we get a
> 'java.lang.OutOfMemoryError: Java heap space'.  This is repeatable but
@
> different time intervals.
> 
> I upgraded to jcs 1.2.6.8 (we were using 1.2.6.5) and we have the same
> issue.
> 
> While reading around I found a claim on the ehcache site that it could
> reproduce this in a test.  So I took them up on it and converted their
> test to run against JCS and I got a 'java.lang.OutOfMemoryError: Java
heap
> space'.
> 
> Below I have pasted in the code of my test.  I know it is a bit exteme
but
> it does prove a point.
> 
> Has anybody else come across this problem, and if so what are my
options
> here, is there something that I could change in my config to stop
this. or
> is it just go into the code a find why the heap cannot clear itself
> through garbage collection.
> 
> I have also pasted the content of my cache.ccf at the end of the
email.
> 
> Any help would be greatly appreciated.
> 
> Owen
> 
>
########################################################################
##
> package com.bgt;
> 
> import java.util.ArrayList;
> import java.util.List;
> 
> import junit.framework.TestCase;
> 
> import org.apache.commons.logging.Log;
> import org.apache.commons.logging.LogFactory;
> import org.apache.jcs.JCS;
> import org.apache.jcs.engine.stats.behavior.IStatElement;
> import org.apache.jcs.engine.stats.behavior.IStats;
> 
> 
> public class JCSTest extends TestCase {
> 
> 	private static final Log LOG =
> LogFactory.getLog(JCSTest.class.getName());
> 
> 	protected JCS jcs;
> 
> 	public static void main(String[] args) {
> 		junit.textui.TestRunner.run(JCSTest.class);
> 	}
> 
> 	public JCSTest(String arg0) {
> 		super(arg0);
> 	}
> 
> 	protected void setUp() throws Exception {
> 		super.setUp();
> 		JCS.setConfigFilename("/cache.ccf");
> 		jcs = JCS.getInstance("testcache");
> 	}
> 
> 	protected void tearDown() throws Exception {
> 		super.tearDown();
> 		jcs.clear();
> 		jcs.dispose();
> 	}
> 
>     /**
>      * Tests adding an entry.
>      */
>     public void testPut() throws Exception {
>         final String value = "value";
>         final String key = "key";
> 
>         // Make sure the element is not found
>         assertEquals(0, getListSize());
> 
>         assertNull(jcs.get(key));
> 
>         jcs.put(key, value);
> 
>         // Get the element
>         LOG.info("jcs.getStats(): " + jcs.getStatistics());
>         assertEquals(1, getListSize());
>         assertNotNull(jcs.get(key));
>         assertEquals(value, jcs.get(key));
>     }
> 
>     /**
>      * Test elements can be removed from the store
>      */
>     public void testRemove() throws Exception {
>         jcs.put("key1", "value1");
>         assertEquals(1, getListSize());
> 
>         jcs.remove("key1");
>         assertEquals(0, getListSize());
> 
>         jcs.put("key2", "value2");
>         jcs.put("key3", "value3");
>         assertEquals(2, getListSize());
> 
>         jcs.remove("key2");
>         assertEquals(1, getListSize());
> 
>         // Try to remove an object that is not there in the store
>         jcs.remove("key4");
>         assertEquals(1, getListSize());
>     }
> 
>     public void testMemortLeak() throws Exception {
>     	long differenceMemoryCache = thrashCache();
>     	LOG.info("Memory Difference is: " + differenceMemoryCache);
>         assertTrue(differenceMemoryCache < 500000);
>     }
> 
>     protected long thrashCache() throws Exception {
> 
>         long startingSize = measureMemoryUse();
>         LOG.info("Memory Used is: " + startingSize);
> 
>         final String value = "value";
>         final String key = "key";
> 
>         // Add the entry
>         jcs.put(key, value);
> 
>         // Create 15 threads that read the keys;
>         final List <JCSTest.Executable> executables = new ArrayList
> <JCSTest.Executable> ();
>         for (int i = 0; i < 15; i++) {
>             final JCSTest.Executable executable = new
JCSTest.Executable()
> {
>                 public void execute() throws Exception {
>                     for (int i = 0; i < 500; i++) {
>                         final String key = "key" + i;
>                         jcs.get(key);
>                     }
>                     jcs.get("key");
>                 }
>             };
>             executables.add(executable);
>         }
>         //Create 15 threads that are insert 500 keys with large byte[]
as
> values
>         for (int i = 0; i < 15; i++) {
>             final JCSTest.Executable executable = new
JCSTest.Executable()
> {
>                 public void execute() throws Exception {
> 
>                     // Add a bunch of entries
>                     for (int i = 0; i < 500; i++) {
>                         // Use a random length value
>                         final String key = "key" + i;
>                         byte[] value = new byte[10000];
>                         jcs.put(key, value);
>                     }
>                 }
>             };
>             executables.add(executable);
>         }
> 
>         runThreads(executables);
>         jcs.clear();
> 
>         long finishingSize = measureMemoryUse();
>         LOG.info("Memory Used is: " + finishingSize);
>         return finishingSize - startingSize;
>     }
> 
>     /**
>      * Runs a set of threads, for a fixed amount of time.
>      */
>     protected void runThreads(final List <JCSTest.Executable>
executables)
> throws Exception {
> 
>         final long endTime = System.currentTimeMillis() + 10000;
>         final Throwable[] errors = new Throwable[1];
> 
>         // Spin up the threads
>         final Thread[] threads = new Thread[executables.size()];
>         for (int i = 0; i < threads.length; i++) {
>             final JCSTest.Executable executable = executables.get(i);
>             threads[i] = new Thread() {
>                 public void run() {
>                     try {
>                         // Run the thread until the given end time
>                         while (System.currentTimeMillis() < endTime) {
>                             executable.execute();
>                         }
>                     } catch (Throwable t) {
>                         // Hang on to any errors
>                         errors[0] = t;
>                     }
>                 }
>             };
>             threads[i].start();
>         }
> 
>         // Wait for the threads to finish
>         for (int i = 0; i < threads.length; i++) {
>             threads[i].join();
>         }
> 
>         // Throw any error that happened
>         if (errors[0] != null) {
>             throw new Exception("Test thread failed.", errors[0]);
>         }
>     }
> 
>     /**
>      * Measure memory used by the VM.
>      *
>      * @return
>      * @throws InterruptedException
>      */
>     protected long measureMemoryUse() throws InterruptedException {
>         System.gc();
>         Thread.sleep(3000);
>         System.gc();
>         return Runtime.getRuntime().totalMemory() -
> Runtime.getRuntime().freeMemory();
>     }
> 
> 
>     /**
>      * A runnable, that can throw an exception.
>      */
>     protected interface Executable {
>         /**
>          * Executes this object.
>          *
>          * @throws Exception
>          */
>         void execute() throws Exception;
>     }
> 
>     private int getListSize() {
>     	final String listSize = "List Size";
>     	final String lruMemoryCache = "LRU Memory Cache";
>     	String result = "0";
>     	IStats istats[] = jcs.getStatistics().getAuxiliaryCacheStats();
>     	for (int i = 0; i < istats.length; i++) {
>     		IStatElement statElements[] =
istats[i].getStatElements();
>     		if (lruMemoryCache.equals(istats[i].getTypeName())) {
>             	for (int j = 0; j < statElements.length; j++) {
>             		if (listSize.equals(statElements[j].getName()))
{
>             			result = statElements[j].getData();
>             		}
>             	}
>     		}
> 
>     	}
>     	return Integer.parseInt(result);
>     }
> 
>     private int getMapSize() {
>     	final String listSize = "Map Size";
>     	String result = "0";
>     	IStatElement statElements[] =
jcs.getStatistics().getStatElements();
>     	for (int i =0; i < statElements.length; i++) {
>     		if (listSize.equals(statElements[i].getName())) {
>     			result = statElements[i].getData();
>     		}
>     	}
>     	return Integer.parseInt(result);
>     }
> }
> 
>
########################################################################
##
> #########
> # DEFAULT CACHE REGION
> 
> jcs.default=LJG
> #jcs.default=
>
jcs.default.cacheattributes=org.apache.jcs.engine.CompositeCacheAttribut
es
> jcs.default.cacheattributes.MaxObjects=10000
>
jcs.default.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory
.l
> ru.LRUMemoryCache
> jcs.default.cacheattributes.UseMemoryShrinker=false
> jcs.default.cacheattributes.MaxMemoryIdleTimeSeconds=3600
> jcs.default.cacheattributes.ShrinkerIntervalSeconds=60
> 
> jcs.default.elementattributes=org.apache.jcs.engine.ElementAttributes
> jcs.default.elementattributes.IsEternal=false
> jcs.default.elementattributes.MaxLifeSeconds=3600
> jcs.default.elementattributes.IdleTime=1800
> jcs.default.elementattributes.IsSpool=true
> jcs.default.elementattributes.IsRemote=true
> jcs.default.elementattributes.IsLateral=true
> 
> jcs.auxiliary.LJG=org.apache.jcs.auxiliary.lateral.LateralCacheFactory
>
jcs.auxiliary.LJG.attributes=org.apache.jcs.auxiliary.lateral.LateralCac
he
> Attributes
> jcs.auxiliary.LJG.attributes.TransmissionTypeName=JAVAGROUPS
> jcs.auxiliary.LJG.attributes.PutOnlyMode=true
> jcs.auxiliary.LJG.attributes.TcpListenerPort=5000
>
jcs.auxiliary.LJG.attributes.JGChannelProperties=UDP(mcast_addr=224.10.1
0.
>
10;mcast_port=5555;ip_ttl=32):PING(timeout=3000;num_initial_members=6):F
D(
>
timeout=3000):VERIFY_SUSPECT(timeout=1500):pbcast.NAKACK(gc_lag=10;retra
ns
>
mit_timeout=600,1200,2400,4800):UNICAST(timeout=600,1200,2400,4800):pbca
st
>
.STABLE(desired_avg_gossip=10000):FRAG:pbcast.GMS(join_timeout=5000;join
_r
> etry_timeout=2000;shun=false;print_local_addr=true)
> 
> 
> ##############################################################
> ################## OPTIONAL THREAD POOL CONFIGURATION
###################
> # Default thread pool config
> thread_pool.default.boundarySize=2000
> thread_pool.default.maximumPoolSize=150
> thread_pool.default.minimumPoolSize=4
> thread_pool.default.keepAliveTime=350000
> #RUN ABORT WAIT BLOCK DISCARDOLDEST
> thread_pool.default.whenBlockedPolicy=RUN
> thread_pool.default.startUpSize=4
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jcs-users-help@jakarta.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org


Mime
View raw message