jakarta-jcs-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Youngho Cho" <youn...@nannet.co.kr>
Subject Re: IndexedDiskCache insert, remove and reinsert question
Date Mon, 17 Nov 2008 00:50:31 GMT
Hello Aaron,

Thanks for the clarification.

Therefore If I want to put some items into a disk as a batch mode without a loss of item,
I have to enough purgatory space which has alway some margin between writing speed and input
speed, right ?

Becasue I have not enough memory for purgatory,  
I hope to controll batch input speed by mornitoring current available Purgatory item count

( Something like  MaxPurgatorySize - Current Purgatory item count ).
And if the purgatory has space than input immediately, but if there is no space in purgatory,
than wait until the purgatory has some space.

Than if purgatory has not enough space, there will be some cache speed degradation but never
loss of item.

But I don't know how to get current purgatory item count.
Is there a method or a simple way to know the value ?
Or I am in totally wrong way?


Thanks,


Youngho






----- Original Message ----- 
From: "Aaron Smuts" <asmuts@yahoo.com>
To: "JCS Users List" <jcs-users@jakarta.apache.org>
Sent: Monday, November 17, 2008 8:22 AM
Subject: Re: IndexedDiskCache insert, remove and reinsert question


> This question has been answered several times recently in one form or another.  
> 
> Purgatory is a buffer on top of the disk.  If you put more items into the buffer than
it allows, the oldest are discarded.  
> 
> The test you are running puts 2000 items into the cache very quickly, more quickly than
they can be written to disk.  If you set the buffer size limit (MaxPurgatorySize) to less
than 2000, then some of the items will be discarded before making it to disk.  
> 
> Aaron
> 
> 
> --- On Wed, 11/12/08, Youngho Cho <youngho@nannet.co.kr> wrote:
> 
>> From: Youngho Cho <youngho@nannet.co.kr>
>> Subject: Re: IndexedDiskCache insert, remove and reinsert question
>> To: "JCS Users List" <jcs-users@jakarta.apache.org>, asmuts@yahoo.com
>> Date: Wednesday, November 12, 2008, 6:59 PM
>> Hello Aaron,
>> 
>> Thanks for your kind consideration.
>> 
>> I tested
>> 
>> http://svn.apache.org/viewvc/jakarta/jcs/trunk/src/test/org/apache/jcs/auxiliary/disk/indexed/IndexedDiskCacheNoMemoryUnitTest.java
>> 
>> with MaxPurgatorySize set 1950 
>> jcs.auxiliary.indexedDiskCache.attributes.MaxPurgatorySize=1950
>> and got FAIL result.
>> 
>> If I set MaxPurgatorySize over 2000, Than I get PASS
>> always. 
>> 
>> 
>> I don't know it is JCS feature or a Bug.
>> 
>> Please refer to the attached Unit Test Result and jcs.log 
>> 
>> 
>> Thanks,
>> 
>> Youngho
>> 
>> ----- Original Message ----- 
>> From: "Aaron Smuts" <asmuts@yahoo.com>
>> To: "JCS Users List"
>> <jcs-users@jakarta.apache.org>
>> Sent: Tuesday, November 11, 2008 10:04 AM
>> Subject: Re: IndexedDiskCache insert, remove and reinsert
>> question
>> 
>> 
>> > Send the contents of the getStats() call.  Are there
>> any errors in the logs?
>> > 
>> > Aaron
>> > 
>> > 
>> > --- On Fri, 11/7/08, Al Forbes
>> <forbes.al@googlemail.com> wrote:
>> > 
>> >> From: Al Forbes <forbes.al@googlemail.com>
>> >> Subject: Re: IndexedDiskCache insert, remove and
>> reinsert question
>> >> To: "JCS Users List"
>> <jcs-users@jakarta.apache.org>
>> >> Date: Friday, November 7, 2008, 3:34 AM
>> >> Hi Youngho,
>> >> 
>> >> I do not have time to look at this in detail, but
>> there are
>> >> many test cases
>> >> in the unit tests. If you have not look there,
>> then
>> >> tt's probably a good
>> >> idea to look at those first.
>> >> 
>> >>
>> http://svn.apache.org/viewvc/jakarta/jcs/trunk/src/test/
>> >> 
>> >> Regards
>> >> Al
>> >> 
>> >> 2008/11/6 Youngho Cho
>> <youngho@nannet.co.kr>
>> >> 
>> >> > Hello,
>> >> >
>> >> > I tested some objects put into
>> IndexedDiskCache and
>> >> remove and reinsert
>> >> > test like following.
>> >> > But I can not pass the test.
>> >> >
>> >> > Is there something wrong in cache.ccf file ?
>> >> > How can I pass the test ?
>> >> >
>> >> >
>> >> >
>> >> > Thanks,
>> >> >
>> >> > Youngho
>> >> >
>> >> >
>> >> >
>> >> > 1. test cache.ccf
>> >> >
>> >> > # DEFAULT CACHE REGION
>> >> > jcs.default=DC
>> >> >
>> >>
>> jcs.default.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
>> >> > jcs.default.cacheattributes.MaxObjects=0
>> >> >
>> >> >
>> >>
>> jcs.default.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache
>> >> >
>> >>
>> jcs.default.cacheattributes.DiskUsagePatternName=UPDATE
>> >> >
>> >>
>> jcs.default.elementattributes=org.apache.jcs.engine.ElementAttributes
>> >> >
>> jcs.default.elementattributes.IsEternal=false
>> >> >
>> >> >
>> >> > # SYSTEM GROUP ID CACHE
>> >> > jcs.system.groupIdCache=DC
>> >> >
>> >> >
>> >>
>> jcs.system.groupIdCache.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
>> >> >
>> jcs.system.groupIdCache.cacheattributes.MaxObjects=0
>> >> >
>> >> >
>> >>
>> jcs.system.groupIdCache.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache
>> >> >
>> >> >
>> >> > # AVAILABLE AUXILIARY CACHES
>> >> >
>> >> >
>> >>
>> jcs.auxiliary.DC=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheFactory
>> >> >
>> >> >
>> >>
>> jcs.auxiliary.DC.attributes=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheAttributes
>> >> > jcs.auxiliary.DC.attributes.DiskPath=d:/tmp
>> >> >
>> jcs.auxiliary.DC.attributes.maxKeySize=1000000
>> >> >
>> jcs.auxiliary.DC.attributes.MaxPurgatorySize=100
>> >> >
>> jcs.auxiliary.DC.attributes.OptimizeOnShutdown=false
>> >> >
>> >> > # PRE-DEFINED CACHE REGIONS
>> >> > jcs.region.testCache1=DC
>> >> >
>> >> >
>> >>
>> jcs.region.testCache1.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
>> >> >
>> jcs.region.testCache1.cacheattributes.MaxObjects=0
>> >> >
>> >> >
>> >>
>> jcs.region.testCache1.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache
>> >> >
>> >> >
>> >> >
>> >>
>> jcs.region.testCache1.elementattributes=org.apache.jcs.engine.ElementAttributes
>> >> >
>> >>
>> jcs.region.testCache1.elementattributes.IsEternal=false
>> >> >
>> >>
>> jcs.region.testCache1.elementattributes.MaxLifeSeconds=0
>> >> >
>> jcs.region.testCache1.elementattributes.IsSpool=true
>> >> >
>> jcs.region.testCache1.elementattributes.IsLateral=true
>> >> >
>> jcs.region.testCache1.elementattributes.IsRemote=true
>> >> >
>> >> >
>> >> >
>> >> > 2. test class
>> >> >
>> >> > //
>> >> > // JCSTest .java
>> >> > //
>> >> >
>> >> > import java.io.File;
>> >> >
>> >> > import junit.framework.TestCase;
>> >> >
>> >> > import org.apache.jcs.JCS;
>> >> > import
>> org.apache.jcs.access.exception.CacheException;
>> >> > import
>> org.apache.jcs.engine.CompositeCacheAttributes;
>> >> > import
>> >>
>> org.apache.jcs.engine.behavior.ICompositeCacheAttributes;
>> >> > import org.apache.jcs.utils.struct.LRUMap;
>> >> >
>> >> > public class JCSTest extends TestCase
>> >> > {
>> >> >
>> >> >    protected static int MAX_NUM = 100000;
>> >> >
>> >> >    protected JCS jcs = null;
>> >> >
>> >> >    protected void setUp() throws Exception
>> >> >    {
>> >> >        File file = new
>> File("d:/tmp");
>> >> >        if(file.exists())
>> >> >        {
>> >> >            File[] files = file.listFiles();
>> >> >            for(int i=0; i < files.length;
>> i++)
>> >> >            {
>> >> >                files[i].delete();
>> >> >            }
>> >> >        }
>> >> >        JCS.setConfigFilename(
>> "/TestJCS.ccf"
>> >> );
>> >> >        jcs =
>> JCS.getInstance("testCache1");
>> >> >    }
>> >> >
>> >> >
>> >> >    protected void tearDown()
>> >> >        throws Exception
>> >> >    {
>> >> >        jcs.dispose();
>> >> >    }
>> >> >
>> >> >    protected void insert(int key) throws
>> >> CacheException
>> >> >    {
>> >> >        Object obj = getCachedObject(key);
>> >> >        jcs.put(new Integer(key), obj);
>> >> >    }
>> >> >
>> >> >    protected void insertMany(int maxNum)
>> throws
>> >> CacheException
>> >> >    {
>> >> >        for (int i = 0; i < maxNum; i++)
>> >> >        {
>> >> >            insert(i);
>> >> >            if ((i % 20000) == 0)
>> >> >            {
>> >> >                try
>> >> >                {
>> >> >                    Thread.sleep(20000);
>> >> >                }
>> >> >                catch(Exception e)
>> >> >                {
>> >> >                    //
>> >> >                }
>> >> >            }
>> >> >        }
>> >> >    }
>> >> >
>> >> >    protected Object get(int key)
>> >> >    {
>> >> >        return jcs.get(new Integer(key));
>> >> >    }
>> >> >
>> >> >    protected void getMany(int maxNum)
>> >> >    {
>> >> >        for (int i = 0; i < maxNum; i++)
>> >> >        {
>> >> >           
>> >> assertNotNull(getCachedObject(i).toString(),
>> get(i));
>> >> >        }
>> >> >    }
>> >> >
>> >> >    protected void testMany(int maxNum)
>> throws
>> >> Exception
>> >> >    {
>> >> >        for (int i = 0; i < maxNum; i++)
>> >> >        {
>> >> >            final MockCache obj =
>> (MockCache)get(i);
>> >> >           
>> >> assertNotNull(getCachedObject(i).toString(),
>> obj);
>> >> >           
>> assertEquals(getCachedObject(i).getValue(),
>> >> obj.getValue());
>> >> >            // remove
>> >> >            jcs.remove(new Integer(i));
>> >> >            assertNull("[" + i 
>> +"]
>> >> should be removed" , get(i));
>> >> >
>> >> >            // reinsert again
>> >> >            insert(i);
>> >> >
>> >> >            final MockCache obj1 =
>> (MockCache)get(i);
>> >> >            // retest
>> >> >           
>> assertEquals(getCachedObject(i).getValue(),
>> >> obj.getValue(),
>> >> > obj1.getValue());
>> >> >        }
>> >> >    }
>> >> >
>> >> >    protected void printStats()
>> >> >    {
>> >> >        System.out.println(jcs.getStats());
>> >> >    }
>> >> >
>> >> >    public void testJCS() throws Exception
>> >> >    {
>> >> >            long start =
>> System.currentTimeMillis();
>> >> >            insertMany(MAX_NUM);
>> >> >            System.out.println("
>> ");
>> >> >            System.out.println("[DONE] :
>> insert
>> >> takes " +
>> >> > (System.currentTimeMillis() - start ) +
>> "msec
>> >> ");
>> >> >
>> >> >            start =
>> System.currentTimeMillis();
>> >> >            getMany(MAX_NUM);
>> >> >            System.out.println("
>> ");
>> >> >            System.out.println("[DONE] :
>> get takes
>> >> " +
>> >> > (System.currentTimeMillis() - start ) +
>> "msec
>> >> ");
>> >> >
>> >> >            start =
>> System.currentTimeMillis();
>> >> >            testMany(MAX_NUM);
>> >> >            System.out.println("
>> ");
>> >> >            System.out.println("[DONE] :
>> test
>> >> takes " +
>> >> > (System.currentTimeMillis() - start) +
>> "msec
>> >> ");
>> >> >
>> >> >            printStats();
>> >> >    }
>> >> >
>> >> >    protected static MockCache
>> getCachedObject(int i)
>> >> >    {
>> >> >        return new
>> MockCache(Integer.toString(i),
>> >> >                "some string [" +
>> >> Integer.toString(i) + "]");
>> >> >    }
>> >> > }
>> >> >
>> >> > //
>> >> > // MockCache.java
>> >> > //
>> >> > import java.io.Serializable;
>> >> >
>> >> > public class MockCache implements
>> Serializable
>> >> > {
>> >> >
>> >> >    private String key = null;
>> >> >    private String value = null;
>> >> >
>> >> >    /**
>> >> >     *
>> >> >     */
>> >> >    public MockCache()
>> >> >    {
>> >> >    }
>> >> >
>> >> >    /**
>> >> >     *
>> >> >     */
>> >> >    public MockCache(String key, String
>> value)
>> >> >    {
>> >> >        this.key = key;
>> >> >        this.value = value;
>> >> >    }
>> >> >
>> >> >    public String getValue()
>> >> >    {
>> >> >        return this.value;
>> >> >    }
>> >> >
>> >> >    public String toString()
>> >> >    {
>> >> >        return "{[" + this.key +
>> "]
>> >> " + this.value + "}";
>> >> >    }
>> >> > }
>> > 
>> >
>> ---------------------------------------------------------------------
>> > To unsubscribe, e-mail:
>> jcs-users-unsubscribe@jakarta.apache.org
>> > For additional commands, e-mail:
>> jcs-users-help@jakarta.apache.org---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> jcs-users-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail:
>> jcs-users-help@jakarta.apache.org
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jcs-users-help@jakarta.apache.org
Mime
View raw message